-- Logs begin at Mon 2022-07-18 09:43:30 UTC, end at Mon 2022-07-18 11:03:12 UTC. -- Jul 18 10:21:25.050371 dv-cinder-846171-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 10:21:25.061160 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 10:21:25 2022] *** Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: nodename: dv-cinder-846171-2-iscsi Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: machine: x86_64 Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: clock source: unix Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: pcre jit disabled Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: detected number of CPU cores: 4 Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: current working directory: / Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: detected binary path: /usr/bin/uwsgi-core Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: your processes number limit is 47680 Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: your memory page size is 4096 bytes Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: detected max file descriptor number: 2048 Jul 18 10:21:25.063194 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: lock engine: pthread robust mutexes Jul 18 10:21:25.064660 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: thunder lock: enabled Jul 18 10:21:25.064660 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 10:21:25.064660 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 10:21:25.103812 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: Python main interpreter initialized at 0x55e8fa4d49c0 Jul 18 10:21:25.103812 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: python threads support enabled Jul 18 10:21:25.103812 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: your server socket listen backlog is limited to 100 connections Jul 18 10:21:25.103812 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: your mercy for graceful operations on workers is 90 seconds Jul 18 10:21:25.104154 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 10:21:25.104237 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: *** Operational MODE: preforking *** Jul 18 10:21:25.104502 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: *** uWSGI is running in multiple interpreter mode *** Jul 18 10:21:25.104502 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: spawned uWSGI master process (pid: 80285) Jul 18 10:21:25.104777 dv-cinder-846171-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 10:21:25.108313 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: spawned uWSGI worker 1 (pid: 80296, cores: 1) Jul 18 10:21:25.108313 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: spawned uWSGI worker 2 (pid: 80297, cores: 1) Jul 18 10:21:25.108313 dv-cinder-846171-2-iscsi devstack@keystone.service[80285]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 10:21:30.620801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:21:30.620801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: from cryptography.utils import int_from_bytes Jul 18 10:21:30.621588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:21:30.621588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: from cryptography.utils import int_from_bytes Jul 18 10:21:30.623384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:21:30.623384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Jul 18 10:21:30.623831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:21:30.623831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Jul 18 10:21:30.671209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 10:21:30.671417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 10:21:30.671509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 10:21:30.671599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] command line args: [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 10:21:30.671708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 10:21:30.671869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 10:21:30.671958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] command line args: [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 10:21:30.672035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ================================================================================ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 10:21:30.672156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 10:21:30.672300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.672386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ================================================================================ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 10:21:30.672550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_dir = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.672634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] admin_token = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.672771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.672860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_dir = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.672944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_source = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] debug = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_source = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] debug = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.673926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_config_append = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_config_append = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.674987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_dir = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_dir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.675928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.676873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.677878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.678742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] publish_errors = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] publish_errors = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.679965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] standard_threads = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.680899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] standard_threads = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_journal = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.681902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_json = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.682017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_journal = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.682125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_stderr = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.682242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_json = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.682363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_syslog = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.682489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_stderr = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.682736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.683344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_syslog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.683344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:21:30.683565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.683565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.683909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.683909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.684984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.685940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.686987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687689 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.687921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.688993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.689984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.690900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691484 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.691975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.692930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.693878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.694934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.695991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.696965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697437 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.697912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.698903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.699918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.700893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.701962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.702903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.703941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.704919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.705979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.706791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.707977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.external = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.external = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.708948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.password = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.password = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.709921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.token = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.token = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.710977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.711935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.712894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.713925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.714894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.715963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.716963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.717923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.718971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.719919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.720896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.721955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.722946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.723885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.724978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.725922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.726988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.727962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.728921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.729049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.729168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.729284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.729399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.730967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.731785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.732968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.733911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.734959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.735931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.736899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737675 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.737906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.738963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.739901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.740893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.741961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.742909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.743975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.744936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.745051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.745216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.745853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.745973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.746910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.747827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.748938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.749880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.750951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.751889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.752964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.753939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.754877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.755970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.756977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.757886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758593 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.758941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.759891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:21:30.760110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 10:21:30.790552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.790766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.791012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.791813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.792629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.792887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.800061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.801287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.828401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.828626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.829822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.830060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.970844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:30.971027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.020402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.020634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.021229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.021472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.084637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.084906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.084998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.085995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.111387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.111622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.111716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.111848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.111950 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.112982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.124653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.124780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.137335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.137576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.137670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.137800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.137884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.138888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:21:31.156468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.156698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:21:31.157039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.157241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:21:31.157510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.157772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.158032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.158173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.158289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.158368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:21:31.158698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.158892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:21:31.159155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.159406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.159655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.159904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.169576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.169854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.170123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.170404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.171276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.171549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.171811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.172089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.178564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.180159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.181884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.182165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.182447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.183367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.183652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.183916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.184010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.184191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.184273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.184466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.184547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.184767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.184874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.185041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.185154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.185307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.185399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.185661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.185931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.186195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.186461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.186720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.213016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.214136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.214891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.215170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.215748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.216021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.220453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.220763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.221085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.221309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.221581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.221857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.228343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.228625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.229108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.229387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.233411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.233702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.233986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.234066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.234270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.234364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.234548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.234630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.234825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.234906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.235093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.235174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.235361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.235442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.235631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.235889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.254822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.254983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.255103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.255191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.259638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.259737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.264063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.264176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.264364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.264514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.269420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.269573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.273090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.273225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.276943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.277953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.278078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.278216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.278351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.278478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.297602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.297747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.297979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.299817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.299970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.300109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.300190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.300262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.306536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.306669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.308902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.309002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.309172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.309256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.312267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.312367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.312555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.312637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.312851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.312947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.313140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.313223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.313419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.313501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.327626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.327728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.327901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.327983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.328174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.329986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.349049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.349154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.352277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.352362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.355718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.356057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.357324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.357715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.358945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.359220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.359358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.359500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.359629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.359906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.366920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.367423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.368561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.369055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.372321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.372602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.372904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.373199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.373289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.373537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.373821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.374098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.382344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.382623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.382916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.383537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.383822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.384102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.391265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.391540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.391814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.392081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.392344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.392531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:21:31.392621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.392837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.392924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.393932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.394991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.395265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.395541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.395805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.396068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:21:31.435357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e8fa4d49c0 pid: 80297 (default app) Jul 18 10:21:31.436835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e8fa4d49c0 pid: 80296 (default app) Jul 18 10:21:31.438778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142021f9-8a61-4519-b5c5-26d79347ac30 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:31.439077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142021f9-8a61-4519-b5c5-26d79347ac30 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:31.439368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142021f9-8a61-4519-b5c5-26d79347ac30 None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:31.443597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Mon Jul 18 10:21:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:40.047384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:40.047898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:40.047979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:40.052274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:21:40.058196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:40.058687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:40.058770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:40.127499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: INFO dbcounter [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] Registered counter for database keystone Jul 18 10:21:40.148801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG oslo_db.sqlalchemy.engines [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80297) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 10:21:40.155221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writer thread running {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 10:21:40.332808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80297) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 10:21:40.333168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80297) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 10:21:40.388334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:40.390006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:40.515075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 18 10:21:40.677762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:40.747915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: INFO dbcounter [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] Registered counter for database keystone Jul 18 10:21:40.766909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG oslo_db.sqlalchemy.engines [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80296) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 10:21:40.773152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writer thread running {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 10:21:40.898368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511aa69-eff2-46c8-8a2c-20fbe623bf83 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:40.903367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:40.904026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:40.904315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:40.904597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:40.912768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:40.964330 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING keystone.common.rbac_enforcer.enforcer [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 10:21:40.964623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206939ec-3242-43b1-bad7-ca35ceb801ed admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:40.974208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Mon Jul 18 10:21:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:21:40.980901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:40.995558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de6d6a-e7ca-40d9-bc56-c50d4b6b72f5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:41.000003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:41.001163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:41.001459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:41.001742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] PATH_INFO: `/v3/projects/1a98445bbc6b4a039db83aaf6d202161` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:41.003251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] RBAC: Authorizing `identity:get_project(project_id=1a98445bbc6b4a039db83aaf6d202161)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:41.056037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 10:21:41.056390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb562b-3f31-4a8c-8c95-0dba68dd4714 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:41.061337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Mon Jul 18 10:21:40 2022] GET /identity/v3/projects/1a98445bbc6b4a039db83aaf6d202161 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:43.055315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4eaf-c9b3-4754-baad-05a65a02d811 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.055826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4eaf-c9b3-4754-baad-05a65a02d811 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.055908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4eaf-c9b3-4754-baad-05a65a02d811 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.060666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:21:43.066271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.066678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.066920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.085778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:43.087321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.095968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 10:21:43.117860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4eaf-c9b3-4754-baad-05a65a02d811 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:43.131301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4eaf-c9b3-4754-baad-05a65a02d811 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:43.135518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4694-b79d-42ea-b5ac-7b4f832431d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:43.136246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4694-b79d-42ea-b5ac-7b4f832431d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.136553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4694-b79d-42ea-b5ac-7b4f832431d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.136868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4694-b79d-42ea-b5ac-7b4f832431d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.144402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4694-b79d-42ea-b5ac-7b4f832431d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:43.146209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4694-b79d-42ea-b5ac-7b4f832431d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:43.155730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Mon Jul 18 10:21:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:21:43.161115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:43.172065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054fbbb-12db-41e0-8282-fbfbf7d29d88 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:43.176601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0494c33-4360-46e0-8c18-d7a7eb09c194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:43.177506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0494c33-4360-46e0-8c18-d7a7eb09c194 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.177798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0494c33-4360-46e0-8c18-d7a7eb09c194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.178079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0494c33-4360-46e0-8c18-d7a7eb09c194 admin admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.179480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0494c33-4360-46e0-8c18-d7a7eb09c194 admin admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:43.181366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0494c33-4360-46e0-8c18-d7a7eb09c194 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:43.183043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Mon Jul 18 10:21:43 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:45.288520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd10c0-f2b7-4181-99f3-0634c07991b3 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.289127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd10c0-f2b7-4181-99f3-0634c07991b3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.289127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd10c0-f2b7-4181-99f3-0634c07991b3 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.290667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:45.291009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8fc0-4b90-4776-8ae2-b78ac0ff3394 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.291570 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8fc0-4b90-4776-8ae2-b78ac0ff3394 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.292083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8fc0-4b90-4776-8ae2-b78ac0ff3394 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.293736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:21:45.295558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.295848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.296132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.298679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.298977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.299265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.316204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:45.317467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:45.336572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG passlib.handlers.bcrypt [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80296) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 10:21:45.337012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG passlib.handlers.bcrypt [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80296) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 10:21:45.382382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:45.383977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:45.398152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 7/12] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:21:45.421590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:45.445562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:45.484014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 6/13] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:45.505777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:45.526054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69de69e-339d-4a46-9071-f6b08eeb8da3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:45.560007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:45.560836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.561140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.561415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.574778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:21:45.589781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:45.590671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:21:45.591513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 8/14] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:21:45.597751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:45.611096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cff76-2d08-435d-9f8f-984800e88cc7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:45.613515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:45.650100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:45.651320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.651320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.651320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.684100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:45.701693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14518d-5807-45d4-b1f8-b9bf9ff7fdaf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:45.721842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:45.723477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:45.736459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:45.737169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.737462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.737748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.738509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:45.738827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:45.740216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:45.752979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 9/15] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:21:45.758472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:45.819804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 7/16] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:21:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:21:45.825754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:45.828483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:45.857758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c650585-7090-43dd-93ee-a9120fbac95c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:45.890256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:45.891128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.891414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.891691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.892933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:45.894389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:45.895999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 10/17] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:21:45 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:21:45.902036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:45.916370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:45.924048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563927f-c8c5-4c22-945f-3dbbfe9b5643 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:45.962278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:45.963081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:45.963363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:45.964366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:45.983931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:45.985429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:21:46.005829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:46.006651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:46.007452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:21:46.007452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:21:46.007452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:21:46.007452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:21:46.007452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 8/18] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:21:46.009486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0884a-5c2e-4369-b8f0-3f49b3c954fa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:46.013430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:46.045512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:46.046339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:46.046623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:46.046899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:46.048163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:46.049730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:46.051325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 11/19] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:21:46.057071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:46.069792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:46.114195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63e431-6b1a-476b-9ddb-69c2e7ecec95 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:46.137639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:46.149752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:46.150407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:46.153688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:46.153688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:46.153688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:46.153688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:46.153688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bbb47-e161-4996-a42a-f96177043410 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:46.155880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75874ff7-044d-45b5-98b5-13b959a9ec91 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:46.170625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 9/20] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:21:46.191576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:46.192259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:46.192547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:46.192865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:46.193717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.domain.id=default, user.id=02055864266e49b39ebadd610e2d403c) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:46.194034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:46.195515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005908c-1edb-4a98-9d86-1d2fa49e8bae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:46.203973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 12/21] 10.222.0.47 () {56 vars in 1322 bytes} [Mon Jul 18 10:21:46 2022] GET /identity/v3/role_assignments?user.id=02055864266e49b39ebadd610e2d403c&scope.domain.id=default&role.id=e4cb7641166847b497698b97084ca96b => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:21:48.634872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c57b7-9b09-4466-87bd-e0888fd3fe19 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:48.635974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c57b7-9b09-4466-87bd-e0888fd3fe19 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:48.636065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c57b7-9b09-4466-87bd-e0888fd3fe19 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:48.637972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 10/22] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:48.643540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830777-6947-4c69-84ef-496b3178543c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:48.643954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830777-6947-4c69-84ef-496b3178543c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:48.644119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830777-6947-4c69-84ef-496b3178543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:48.669471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45830777-6947-4c69-84ef-496b3178543c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:48.670678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45830777-6947-4c69-84ef-496b3178543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:48.684662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc1af1-8c16-4291-8f40-5ce98ffb804e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:48.685258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc1af1-8c16-4291-8f40-5ce98ffb804e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:48.685733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc1af1-8c16-4291-8f40-5ce98ffb804e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:48.687303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 11/23] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:48.692672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:48.693218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:48.693688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:48.730913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:48.732459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:48.794055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 13/24] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:48.833798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830777-6947-4c69-84ef-496b3178543c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:48.879344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 12/25] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:48.914765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:48.993057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.021880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.054632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830777-6947-4c69-84ef-496b3178543c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.100962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8e9ff-731f-4eb8-aa2d-d2e55d819d7b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.140398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7598e34-3f34-4ee4-9318-f2bec5176f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.141244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598e34-3f34-4ee4-9318-f2bec5176f4a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.141533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598e34-3f34-4ee4-9318-f2bec5176f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.141817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598e34-3f34-4ee4-9318-f2bec5176f4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.145163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe759567-c835-4def-a270-a797de103a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.145844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759567-c835-4def-a270-a797de103a22 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.146133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759567-c835-4def-a270-a797de103a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.146418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759567-c835-4def-a270-a797de103a22 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.147200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe759567-c835-4def-a270-a797de103a22 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:49.148596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe759567-c835-4def-a270-a797de103a22 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:49.169019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7598e34-3f34-4ee4-9318-f2bec5176f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c7598e34-3f34-4ee4-9318-f2bec5176f4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:21:49.177290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:49.178127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:21:49.178935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 14/26] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:21:49.180337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd859cd-2f5f-411a-8fa6-9586b50e626f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.180337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd859cd-2f5f-411a-8fa6-9586b50e626f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.181583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd859cd-2f5f-411a-8fa6-9586b50e626f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.186049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 15/27] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:49.187381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d2452-6a55-4c29-b889-7fb25dcaf2b9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.187681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d2452-6a55-4c29-b889-7fb25dcaf2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.187974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d2452-6a55-4c29-b889-7fb25dcaf2b9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.189615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 16/28] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:49.190240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d2452-6a55-4c29-b889-7fb25dcaf2b9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:49.311264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d2452-6a55-4c29-b889-7fb25dcaf2b9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.349091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.349768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.350055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.350335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.351095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:49.351407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:49.352809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21af77-d5e1-456f-8b8c-26906cd36a48 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:49.354930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.362947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 17/29] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:49.368731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.369034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.369317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.386979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:49.388213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:49.405171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 13/30] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Jul 18 10:21:48 2022] POST /identity/v3/roles => generated 219 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:21:49.407241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.407745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.408255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.451845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:49.457474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:49.522934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 18/31] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:49.523850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:49.595020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 14/32] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:49.595919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:49.649855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.690205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49280f-b7e7-4ff4-8a04-24ac79b41100 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.729599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.738465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b598c-4419-49d0-8b5e-9ca2339acd70 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.741126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.743184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.743623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.744048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.745667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:49.747295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:49.750405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 19/33] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:21:49 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:49.751351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:49.790552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.792237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.792893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.793759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.795109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:49.797070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:49.872125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 15/34] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Jul 18 10:21:49 2022] POST /identity/v3/roles => generated 225 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:21:49.872328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd3096-cdae-4eeb-9e04-e40db9973148 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.876793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:49.918969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.919877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.920302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.920738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.934261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:49.935838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e293addb-f628-4af1-bcbe-b5ca8dcb16dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:21:49.958696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:49.959723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:49.960538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:21:49.960538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:21:49.960538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:21:49.960538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:21:49.960538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 20/35] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:21:49.961437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519331d5-47af-43d8-800f-fbe5acd97974 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.961884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519331d5-47af-43d8-800f-fbe5acd97974 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.962295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519331d5-47af-43d8-800f-fbe5acd97974 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.964049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 21/36] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:49.969823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519331d5-47af-43d8-800f-fbe5acd97974 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:50.010303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:50.019289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b914af8-8682-4db4-808d-213e6744b50d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.034389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:50.063273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c09375-6793-406d-ac76-e43e8307978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.064088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c09375-6793-406d-ac76-e43e8307978b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.064376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c09375-6793-406d-ac76-e43e8307978b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.064660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c09375-6793-406d-ac76-e43e8307978b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.066177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c09375-6793-406d-ac76-e43e8307978b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.067650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c09375-6793-406d-ac76-e43e8307978b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:50.069403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 16/37] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:50.070840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.071298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.071724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.091411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:50.091411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:50.109791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519331d5-47af-43d8-800f-fbe5acd97974 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.159156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.159829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.160121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.160714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.161188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:50.161505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.162897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb58c69-3af0-4bab-8651-ef5cee3ec26f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:50.174602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 17/38] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:50.175432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:50.185646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 22/39] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:50.185646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4585029-a6e2-402e-8d0d-35a47752f289 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.185646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4585029-a6e2-402e-8d0d-35a47752f289 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.185646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4585029-a6e2-402e-8d0d-35a47752f289 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.185646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 23/40] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:50.187701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4585029-a6e2-402e-8d0d-35a47752f289 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:50.273323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:50.280698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cce5a-8c57-4baf-8688-afffabe0040f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.292497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4585029-a6e2-402e-8d0d-35a47752f289 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.299919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:50.314322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37b980-19ab-421c-96a2-9449b810ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.315224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b980-19ab-421c-96a2-9449b810ea4f None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.315510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b980-19ab-421c-96a2-9449b810ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.315792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b980-19ab-421c-96a2-9449b810ea4f None admin] PATH_INFO: `/v3/domains/default/users/02055864266e49b39ebadd610e2d403c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.320518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b980-19ab-421c-96a2-9449b810ea4f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=02055864266e49b39ebadd610e2d403c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.320518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b980-19ab-421c-96a2-9449b810ea4f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:50.326949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.327647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.327939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.328224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.328977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.330440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:50.454899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 18/41] 10.222.0.47 () {58 vars in 1361 bytes} [Mon Jul 18 10:21:50 2022] PUT /identity/v3/domains/default/users/02055864266e49b39ebadd610e2d403c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:21:50.456176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.456472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.456780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.476160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:50.477444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:50.568848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 19/42] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:50.568848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:50.668302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:50.685409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c9c3e-df06-4b38-bc98-7102819e30db None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.719350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.720149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.720438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.720740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.721986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.723893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:50.729083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 24/43] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:21:50 2022] POST /identity/v3/projects => generated 301 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:21:50.729083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:50.729741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 20/44] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:50.736826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:50.879876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3b444-4cbc-4fce-baa0-515c8e5f9ff9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.886276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0762a-317f-428a-97d1-7672abf0aca9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:50.926253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c42d0-bd5e-4011-bff8-b5abe02d6b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.927395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c42d0-bd5e-4011-bff8-b5abe02d6b04 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.927701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c42d0-bd5e-4011-bff8-b5abe02d6b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.928297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c42d0-bd5e-4011-bff8-b5abe02d6b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.929642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c42d0-bd5e-4011-bff8-b5abe02d6b04 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.930708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c42d0-bd5e-4011-bff8-b5abe02d6b04 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:50.937431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e057e-c336-49f6-a2b5-f7f8a17de3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:50.938138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e057e-c336-49f6-a2b5-f7f8a17de3a1 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:50.938425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e057e-c336-49f6-a2b5-f7f8a17de3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:50.938707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e057e-c336-49f6-a2b5-f7f8a17de3a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:50.939436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e057e-c336-49f6-a2b5-f7f8a17de3a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:50.940903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e057e-c336-49f6-a2b5-f7f8a17de3a1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:51.065722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 25/45] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Jul 18 10:21:50 2022] POST /identity/v3/roles => generated 223 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:21:51.131205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 21/46] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:21:50 2022] POST /identity/v3/projects => generated 312 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:21:52.623362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6618-85de-4ac2-ac7e-27b1ef862df0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:52.623850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6618-85de-4ac2-ac7e-27b1ef862df0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:52.623850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6618-85de-4ac2-ac7e-27b1ef862df0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:52.625270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 26/47] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:52.630318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:52.630754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:52.630911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:52.637665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:52.647624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:52.648852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:52.714895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 22/48] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:21:52.739247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6618-85de-4ac2-ac7e-27b1ef862df0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:52.760101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:52.842162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6618-85de-4ac2-ac7e-27b1ef862df0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:52.875247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:52.875975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:52.876263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:52.876544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:52.886589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:21:52.894209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:52.895226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:21:52.896014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 27/49] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:21:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:21:52.903035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:53.001416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e66e1-8e29-4041-a2d8-f07d3ed235cc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:53.034241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:53.034903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.035188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.035467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.036221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:53.036529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:53.038085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:53.039742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:53.051411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 23/50] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:21:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:53.057011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:53.135203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:53.156271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474ae51-3f1d-4677-b9fd-2c6dd028cf89 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:53.190668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94461b-3827-48e0-89b2-e79d6b6004de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:53.191549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94461b-3827-48e0-89b2-e79d6b6004de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.191841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94461b-3827-48e0-89b2-e79d6b6004de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.192126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94461b-3827-48e0-89b2-e79d6b6004de None admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.193333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94461b-3827-48e0-89b2-e79d6b6004de None admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:53.194807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94461b-3827-48e0-89b2-e79d6b6004de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:53.196460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 28/51] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:21:53 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:53.202521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:53.203315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec7f3a-4349-4a5b-a714-5c4ee6d82ac5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.203610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec7f3a-4349-4a5b-a714-5c4ee6d82ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.203897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec7f3a-4349-4a5b-a714-5c4ee6d82ac5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.205817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 29/52] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:21:53.211387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.211759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.212105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.229154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:53.230685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:53.301215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 30/53] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:21:53.310914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df369-eef1-455d-80e0-768e9659f8e9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:53.324459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:53.348443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:53.349277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.349562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.349845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.351063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:53.352514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:53.354273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 24/54] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:53.359937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:53.398962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:53.424113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cca26d-3de2-46d7-a6a6-afa4563a76e7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:53.463290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:53.464652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.464923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.465207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.466516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:53.468023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:53.469854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 31/55] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:21:53.475652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:53.526831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:53.565439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139f5a9-d454-450d-a89f-a30ab1856a61 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:53.577873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86c398-dddb-41a9-82d1-f3be222e7f6e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:53.599881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:53.600535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.600853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.601132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.601972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.domain.id=default, user.id=02055864266e49b39ebadd610e2d403c) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:53.602286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:53.603810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84c59e-5b57-4360-abb4-d5408c785943 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:53.612022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 25/56] 10.222.0.47 () {56 vars in 1322 bytes} [Mon Jul 18 10:21:53 2022] GET /identity/v3/role_assignments?user.id=02055864266e49b39ebadd610e2d403c&scope.domain.id=default&role.id=e4cb7641166847b497698b97084ca96b => generated 530 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:53.613063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77e822-a20d-405b-a71e-90ba1a53b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:53.613738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e822-a20d-405b-a71e-90ba1a53b5d4 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:53.614034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e822-a20d-405b-a71e-90ba1a53b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:53.614322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e822-a20d-405b-a71e-90ba1a53b5d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:53.615043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77e822-a20d-405b-a71e-90ba1a53b5d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:53.616523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77e822-a20d-405b-a71e-90ba1a53b5d4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:53.632861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:53.773937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 32/57] 10.222.0.47 () {60 vars in 1140 bytes} [Mon Jul 18 10:21:53 2022] POST /identity/v3/projects => generated 298 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:21:55.788939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6088-907d-430b-ac19-e42d153a09a6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:55.789440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6088-907d-430b-ac19-e42d153a09a6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:55.789545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6088-907d-430b-ac19-e42d153a09a6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:55.790853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 26/58] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:55.796761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:55.797155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:55.797351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:55.814491 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:55.815713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:55.880007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 33/59] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:55.904299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6088-907d-430b-ac19-e42d153a09a6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:56.003445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6088-907d-430b-ac19-e42d153a09a6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:56.029238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:56.036627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9dd51-afbf-4d3e-9643-ef8d72095c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:56.037439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9dd51-afbf-4d3e-9643-ef8d72095c4c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:56.037730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9dd51-afbf-4d3e-9643-ef8d72095c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:56.038013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9dd51-afbf-4d3e-9643-ef8d72095c4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:56.045933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9dd51-afbf-4d3e-9643-ef8d72095c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:56.045933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9dd51-afbf-4d3e-9643-ef8d72095c4c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:56.045933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 27/60] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:21:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:21:56.048728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:56.145927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7f53-b53d-4402-80f2-06839b7c71e3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:56.172698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:56.181233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a87d0-a387-41e9-809a-7c230e88099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:56.182449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a87d0-a387-41e9-809a-7c230e88099a None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:56.182737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a87d0-a387-41e9-809a-7c230e88099a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:56.183019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a87d0-a387-41e9-809a-7c230e88099a None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:56.183729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a87d0-a387-41e9-809a-7c230e88099a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:56.197873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a87d0-a387-41e9-809a-7c230e88099a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:56.312186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 34/61] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:21:56 2022] POST /identity/v3/users => generated 275 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:21:59.423878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccc1cc-588a-494d-8235-e78a653d2f36 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.424860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccc1cc-588a-494d-8235-e78a653d2f36 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.425360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccc1cc-588a-494d-8235-e78a653d2f36 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.426930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 28/62] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:59.432867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.433884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.434363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.456953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:59.458415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:59.501164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f9a5b-1184-46ae-9c10-139656e4048a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.501803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f9a5b-1184-46ae-9c10-139656e4048a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.502295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f9a5b-1184-46ae-9c10-139656e4048a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.503881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 29/63] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:59.508771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3132-4075-4200-8109-cbebd6c8702c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.509273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3132-4075-4200-8109-cbebd6c8702c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.509763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3132-4075-4200-8109-cbebd6c8702c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.537907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-93bb3132-4075-4200-8109-cbebd6c8702c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:59.539055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-93bb3132-4075-4200-8109-cbebd6c8702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:59.570257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 35/64] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:59.594256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:59.696998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 30/65] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:21:59.710214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3a122-4a26-448d-af3e-ca8a4f136c50 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:59.724007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d1c79-8de9-4664-89fb-8862006d9d8b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.724303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d1c79-8de9-4664-89fb-8862006d9d8b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.724785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d1c79-8de9-4664-89fb-8862006d9d8b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.725135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:59.726495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 31/66] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:21:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:21:59.732481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672935-e03b-440b-be06-5d64c57207fb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.733080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672935-e03b-440b-be06-5d64c57207fb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.733503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672935-e03b-440b-be06-5d64c57207fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.752834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-55672935-e03b-440b-be06-5d64c57207fb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:59.754367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-55672935-e03b-440b-be06-5d64c57207fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:59.757356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:59.758549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.759042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.759519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.772859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:21:59.781937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:59.782855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:21:59.783637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:21:59.785663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 36/67] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:21:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:21:59.786670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:59.839956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 32/68] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:21:59.841152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672935-e03b-440b-be06-5d64c57207fb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:59.905070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:59.930381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc21b56-3d66-4304-b60b-138290515da3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:59.950778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672935-e03b-440b-be06-5d64c57207fb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:59.967828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:59.968632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.968950 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.969238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.981004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:21:59.986687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:59.987361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:59.987651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:59.987936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:59.988722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:21:59.989043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:21:59.989133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:21:59.989904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:21:59.990648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 37/69] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:21:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:21:59.991254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244956f9-1638-49a7-9b18-25e0df5e697e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:59.991317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.001160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 33/70] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:21:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.003813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99597f2-7330-4692-90ec-e81b73f9c2a2 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.004320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99597f2-7330-4692-90ec-e81b73f9c2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.004853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99597f2-7330-4692-90ec-e81b73f9c2a2 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.006806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 34/71] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:00.007734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99597f2-7330-4692-90ec-e81b73f9c2a2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.027031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.099341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079cc35-8dcd-4d4d-9168-bed9d33c264f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.120981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99597f2-7330-4692-90ec-e81b73f9c2a2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.134490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.136034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.136326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.136599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.139894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.147415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:00.155635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:00.162898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:00.163658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 38/72] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:21:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:00.164276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e536a0-a768-4909-be86-85e372584274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.164367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.164441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e536a0-a768-4909-be86-85e372584274 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.164441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e536a0-a768-4909-be86-85e372584274 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.164441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e536a0-a768-4909-be86-85e372584274 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.164441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e536a0-a768-4909-be86-85e372584274 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:00.164441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e536a0-a768-4909-be86-85e372584274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.164441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e536a0-a768-4909-be86-85e372584274 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.174260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 35/73] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.175608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.175910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.176195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.192866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:00.194073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:00.268926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.272695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 36/74] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:00.277944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd6930-cfcb-4103-9538-ea9a9f25ffe4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.281635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.313968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.314847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.315149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.315433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.329427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.330940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.332912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 39/75] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.337576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.383661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6c6c3-a57a-4cee-9836-e8e3113c6de3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.394034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.419446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.420127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.420416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.420722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.421482 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:00.421796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.423172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d84a35-dc66-42e6-95b4-4ba5fae1fdce None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.436623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 37/76] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.436623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a114b9-ddfa-429b-8864-5b4188143e1c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.436623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a114b9-ddfa-429b-8864-5b4188143e1c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.436623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a114b9-ddfa-429b-8864-5b4188143e1c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.436623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 38/77] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:00.437436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a114b9-ddfa-429b-8864-5b4188143e1c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.441652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f2775-833f-48de-a301-ea2780f51b88 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.475351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.476212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.476500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.476815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.478007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.479599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.481378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 40/78] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.485524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.503869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.537775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a114b9-ddfa-429b-8864-5b4188143e1c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.572086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.572973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.573282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.573579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.583846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91131e3b-afa8-4f1e-a383-3c06d85694e0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.584864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:00.594806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:00.595877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 39/79] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:00.596801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.621795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.622453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.622746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.623032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.627791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.693002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.694405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.699556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a643e7c-8ea8-468f-a11a-d2c0af8f6666 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.713901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.733802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795a0b8-6410-4145-8c24-3b986a775fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.734680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795a0b8-6410-4145-8c24-3b986a775fcc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.734966 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795a0b8-6410-4145-8c24-3b986a775fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.735249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795a0b8-6410-4145-8c24-3b986a775fcc None admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.736478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795a0b8-6410-4145-8c24-3b986a775fcc None admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.738036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795a0b8-6410-4145-8c24-3b986a775fcc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.739686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 40/80] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.740923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.741222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.741504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.758436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:00.759629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:00.799061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 41/81] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:00.800621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.825944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 41/82] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:00.827143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:00.857264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.900851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdf1f5-f0eb-4765-a119-5f05a8fb5112 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.918038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:00.925699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9cb9-d985-4332-b383-0f8f60718f91 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:00.934715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.935373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.935665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.935951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.966361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:00.967025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:00.967333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:00.967611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:00.968398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:00.968776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:00.970221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:00.978722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 42/83] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:00.979541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.006627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.008079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.069502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.365695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dcf94-4c13-47ce-8a0c-3219afb08d73 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.366479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 42/84] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:01.366479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.366657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.366657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.366657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.366657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.366952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.367037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.367037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.367186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f7b53-0454-4209-9463-0bc93b672a2d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.367186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.367186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.367186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.367517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.367517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.367517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.367517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 43/85] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:01.367517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.367807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.367807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 43/86] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:01.367807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.367982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.367982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a825e3-8c03-481d-82e9-b13c5944830f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.390208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f1650-ff1f-4754-9ce6-ddbbf84d5bbe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.390937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.391733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.392022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.392305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.393510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.395047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.401690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 44/87] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:01.401690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.428728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.429606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.429896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.430178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.431377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.433082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.434739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 44/88] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:01.435511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.493702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.498819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe0d2e-a058-4fe1-8e1b-d0cb8a6b547f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.532553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.533006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711eea7-e9ef-4e99-8be7-8681ebd949b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.533445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.533735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.534018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.535203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.536656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.538379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 45/89] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:01.540298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.565690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.568349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.569041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.569371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.569650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.570489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] RBAC: Adding query filter params (role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=4b3ba696b60e4dc9b69780cc894e58a8) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:01.570804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.572247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.578384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 45/90] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/role_assignments?user.id=4b3ba696b60e4dc9b69780cc894e58a8&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:01.579149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.659507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c9698-2ca6-4c3f-814f-2624388abf3c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.698019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.698913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.699199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.699479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.700804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.701787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61853927-7215-4559-ab56-e69e79d164b1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.702303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.709420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 46/91] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:01.709420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.737190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.750560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.754813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.754813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.754813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.754813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.754813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:01.812416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:01.822132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fb45b-85dd-4dac-bab9-c9f9b07e70d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.857637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:01.858298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:01.858589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:01.859198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:01.867292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 46/92] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:01 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:01.867292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:01.974233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:01.974752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed79649-f6fa-42ca-97f0-2f824049c033 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:01.975647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:02.015515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:02.016723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:02.017243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:02.017722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:02.018820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=4b3ba696b60e4dc9b69780cc894e58a8) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:02.019371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:02.021077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:02.028071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 47/93] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/role_assignments?user.id=4b3ba696b60e4dc9b69780cc894e58a8&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=19944ab23542462988b68290167ac622 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:02.029312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:02.099002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:02.101179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:02.114766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 47/94] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:02.125860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:02.178229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac55a7-7505-44ad-af7e-67d3cea79290 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:02.232043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:02.232651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:02.233033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:02.233320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:02.234171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=02055864266e49b39ebadd610e2d403c) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:02.234491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:02.235953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:02.248327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 48/95] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:02 2022] GET /identity/v3/role_assignments?user.id=02055864266e49b39ebadd610e2d403c&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:02.278526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27309dd-7858-4975-9fd8-e4ee451da168 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:02.325322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086d972-ac55-4ed2-976b-c7b51bcfb90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:02.326231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086d972-ac55-4ed2-976b-c7b51bcfb90a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:02.326520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086d972-ac55-4ed2-976b-c7b51bcfb90a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:02.326803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086d972-ac55-4ed2-976b-c7b51bcfb90a None admin] PATH_INFO: `/v3/projects/84f6e66f87dc45f58cba121ce862b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:02.328249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086d972-ac55-4ed2-976b-c7b51bcfb90a None admin] RBAC: Authorizing `identity:get_project(project_id=84f6e66f87dc45f58cba121ce862b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:02.329827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086d972-ac55-4ed2-976b-c7b51bcfb90a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:02.332371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 48/96] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:02 2022] GET /identity/v3/projects/84f6e66f87dc45f58cba121ce862b0f5 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:02.338243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:02.479054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94e410-d9b7-4bc7-99e3-f87852a7ba9d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:02.489072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:02.520399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:02.521334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:02.521444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:02.521656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:02.522519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, scope.project.id=84f6e66f87dc45f58cba121ce862b0f5, user.id=4b3ba696b60e4dc9b69780cc894e58a8) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:02.522846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:02.524308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f9a7-6922-4b53-8bf0-9b4865b55cb9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:02.531598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 49/97] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:02 2022] GET /identity/v3/role_assignments?user.id=4b3ba696b60e4dc9b69780cc894e58a8&scope.project.id=84f6e66f87dc45f58cba121ce862b0f5&role.id=19944ab23542462988b68290167ac622 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:04.285843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef3155-d0eb-4fb8-9367-786faf8ec851 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:04.286315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef3155-d0eb-4fb8-9367-786faf8ec851 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:04.286405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef3155-d0eb-4fb8-9367-786faf8ec851 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:04.287883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 49/98] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:04.293932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:04.294344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:04.294596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:04.327502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:04.329004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:04.418695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 50/99] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:04.448484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef3155-d0eb-4fb8-9367-786faf8ec851 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:04.528334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:04.578448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef3155-d0eb-4fb8-9367-786faf8ec851 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:04.622272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:04.623394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:04.623886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:04.624347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:04.639273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:04.648017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:04.648926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:04.649897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:04.652671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 50/100] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:04.658146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:04.796870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ef24-922a-4c8a-a2f7-903b00b46b96 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:04.798643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:04.839063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:04.839836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:04.845069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:04.845370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:04.846160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:04.846484 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:04.847900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aefb6e-ab1c-4a1c-b724-67c8c9170d9d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:04.856995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 51/101] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:04.862553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:04.921705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341dde-9561-40f6-adad-6963bbb2de2f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:04.922485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341dde-9561-40f6-adad-6963bbb2de2f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:04.923342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341dde-9561-40f6-adad-6963bbb2de2f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:04.925290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 52/102] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:04.930530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091dfa-2c17-472c-8e6c-9399ab3bb5f5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:04.931067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091dfa-2c17-472c-8e6c-9399ab3bb5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:04.932069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e091dfa-2c17-472c-8e6c-9399ab3bb5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:04.961052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6e091dfa-2c17-472c-8e6c-9399ab3bb5f5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:04.962616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6e091dfa-2c17-472c-8e6c-9399ab3bb5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:04.989558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db717f8a-2799-4d27-ab49-04d3bd6ce87d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.011158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.031382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc7cc8-5486-41e7-9494-f926456d59c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:05.032896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7cc8-5486-41e7-9494-f926456d59c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.033354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7cc8-5486-41e7-9494-f926456d59c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.033895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7cc8-5486-41e7-9494-f926456d59c3 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.035276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc7cc8-5486-41e7-9494-f926456d59c3 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:05.037156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc7cc8-5486-41e7-9494-f926456d59c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:05.039499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 51/103] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:04 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:05.041550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f301-fe85-4551-b857-af7093ad82d8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.042316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f301-fe85-4551-b857-af7093ad82d8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.042748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f301-fe85-4551-b857-af7093ad82d8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.044153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 52/104] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:05.048299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f301-fe85-4551-b857-af7093ad82d8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.119621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 53/105] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:05.120968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.121268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.121553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.168515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:05.169755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:05.216305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f301-fe85-4551-b857-af7093ad82d8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.282418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:05.283305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.283596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.283882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.287667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:05.290527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:05.294628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 53/106] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:05 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:05.296930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.303269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 54/107] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:05.304827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.356894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.420746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.467163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a197b1-1af6-4509-9770-4fcb5ff0d2ed None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.494509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8c5fe-b004-485d-8105-649c53318a2b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.505146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d613178-2a5e-4a92-ace3-6fd777d884cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:05.506060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d613178-2a5e-4a92-ace3-6fd777d884cf None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.506362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d613178-2a5e-4a92-ace3-6fd777d884cf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.506602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d613178-2a5e-4a92-ace3-6fd777d884cf None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/4b3ba696b60e4dc9b69780cc894e58a8/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.508867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d613178-2a5e-4a92-ace3-6fd777d884cf None admin] RBAC: Authorizing `identity:create_grant(project_id=61a5cd521b194aab81a0ff7939ebd317, user_id=4b3ba696b60e4dc9b69780cc894e58a8, role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:05.511887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d613178-2a5e-4a92-ace3-6fd777d884cf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:05.530138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:05.534817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.534817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.534817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.541847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:05.549483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:05.550467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 54/108] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:05.551489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.586275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 55/109] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:05 2022] PUT /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/4b3ba696b60e4dc9b69780cc894e58a8/roles/6413d71218d344e8b26c8577cac9c4dc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:05.588412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310fad3-10b5-4b5d-b864-d96a2339e144 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.589527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310fad3-10b5-4b5d-b864-d96a2339e144 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.590084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310fad3-10b5-4b5d-b864-d96a2339e144 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.591717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 56/110] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:05.593554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dd149-ffe9-48ec-a6f6-e41a0a5588aa None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.594123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dd149-ffe9-48ec-a6f6-e41a0a5588aa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.594629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dd149-ffe9-48ec-a6f6-e41a0a5588aa None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.596715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 57/111] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:05.598033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dd149-ffe9-48ec-a6f6-e41a0a5588aa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.674405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e313fc-cb35-41ad-97f4-ce3a826943ff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.691600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.696990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.715247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa987bf-78e0-470d-ab2f-bb9756d16276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:05.716061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa987bf-78e0-470d-ab2f-bb9756d16276 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.716349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa987bf-78e0-470d-ab2f-bb9756d16276 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.716628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa987bf-78e0-470d-ab2f-bb9756d16276 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.717483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dd149-ffe9-48ec-a6f6-e41a0a5588aa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.729352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa987bf-78e0-470d-ab2f-bb9756d16276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ffa987bf-78e0-470d-ab2f-bb9756d16276 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:05.737252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:05.738194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:05.739736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 55/112] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:05.740635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e39f3-cfad-40e9-8eab-742167f9484f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.740635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e39f3-cfad-40e9-8eab-742167f9484f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.740635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e39f3-cfad-40e9-8eab-742167f9484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.759024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-311e39f3-cfad-40e9-8eab-742167f9484f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:05.761175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-311e39f3-cfad-40e9-8eab-742167f9484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:05.761443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:05.762050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.762336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.762611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.764187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:05.764187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:05.765687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da7fd6-2d40-4a1f-905f-33a391788339 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:05.777156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 58/113] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:05.778554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332663-f99b-4348-8454-d417d7da455f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.778854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332663-f99b-4348-8454-d417d7da455f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.779133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332663-f99b-4348-8454-d417d7da455f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.799649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-92332663-f99b-4348-8454-d417d7da455f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:05.800954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92332663-f99b-4348-8454-d417d7da455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:05.839130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 56/114] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:05.840469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683b386-3402-4586-b0ef-97bbd3db7e9c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.840814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683b386-3402-4586-b0ef-97bbd3db7e9c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.841103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683b386-3402-4586-b0ef-97bbd3db7e9c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.846371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 57/115] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:05.846371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683b386-3402-4586-b0ef-97bbd3db7e9c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.886952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 59/116] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:05.886952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92332663-f99b-4348-8454-d417d7da455f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:05.968496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683b386-3402-4586-b0ef-97bbd3db7e9c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:05.972175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.972699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.995895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92332663-f99b-4348-8454-d417d7da455f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.005004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.005683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.005972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.006267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.007292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:06.007830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.009429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540226a0-2d5e-46dd-81cc-59e9322c2824 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.019248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 58/117] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.020654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.020982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.021266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.038211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:06.039410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:06.039599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.040771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.041312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.041765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.043210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.044899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.046813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 60/118] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:05 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.047755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.120774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 59/119] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:06.120774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.167949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e358d1-4eb9-4614-b2d7-5c5031bb1060 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.206042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.206842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.207381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.207673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.208900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.210491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.212135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 61/120] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.214725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.221493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.227510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a006da-2253-4b3f-85a3-18e67297ac77 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.231644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.271325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.272591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.273687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.274047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.289110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:06.297257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:06.298240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:06.299000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 60/121] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:06.300232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.329790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8816d2-8eac-4b25-ae3c-e0d032d1e9d9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.365275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.366154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.366444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.366732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.367951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.369969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.371858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 62/122] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.376820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.410146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334e517-53d5-493d-bbb4-25c89681d791 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.447055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.448190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.448626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.449034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.450336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.452009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.453982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 61/123] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.453982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.473165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.484789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.492881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6fb33-8854-4f0b-9a0a-d588c4f686bf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.529260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.530302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.530597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.530963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.545990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:06.554082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:06.554933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:06.555713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 63/124] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:06.557556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.568312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a07cd-0b59-453f-af2b-666fd2ecf9ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.606801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.607468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.607756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.608042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.608782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.610212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.660743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47f760-44f1-4500-8230-b16752055684 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.695736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 62/125] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:22:06 2022] POST /identity/v3/projects => generated 302 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:06.696515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.697374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.697717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.698005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.699003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:06.699147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.700490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.704154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.711305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 64/126] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.711949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:06.715413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.844697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.852992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084a087-2819-4ff5-98d4-d584cfc66bc2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.872813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af572c92-dcc0-4366-b88f-c17b2adbb065 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:06.902301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.902768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.903183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.903721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/02055864266e49b39ebadd610e2d403c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.906214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] RBAC: Authorizing `identity:create_grant(project_id=61a5cd521b194aab81a0ff7939ebd317, user_id=02055864266e49b39ebadd610e2d403c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.908239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.914394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:06.915203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:06.915492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:06.915774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.917538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:06.919010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:06.921113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 63/127] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:06.921904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.019862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 65/128] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:06 2022] PUT /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/02055864266e49b39ebadd610e2d403c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:07.025974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.049687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dec028-399b-4f5a-b32e-785aee5010d0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:07.105326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:07.137742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:07.142382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.142834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.143503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.144423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:07.146349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:07.147974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:07.170718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 64/129] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:07.179210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31f6d7-f5d0-4e9a-9734-b93ac415614d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:07.180599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.198962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:07.227686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:07.235098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.235098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.235098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] PATH_INFO: `/v3/users/7f086f0d540c4d88934e311e9d38adcd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.250844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] RBAC: Authorizing `identity:get_user(user_id=7f086f0d540c4d88934e311e9d38adcd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:07.251750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:07.253826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 66/130] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:07 2022] GET /identity/v3/users/7f086f0d540c4d88934e311e9d38adcd => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:07.254916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.380497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00107b7-da77-433a-956f-33cbdd768d17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:07.385183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec1e7f-652e-485f-bf71-3e8333a597ab None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:07.410178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:07.427457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:07.428575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.429445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.429863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.431233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:07.434370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:07.437874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 67/131] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:07 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:07.440119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.462004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:07.463530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.463976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.464362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/4b3ba696b60e4dc9b69780cc894e58a8/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.470756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] RBAC: Authorizing `identity:create_grant(project_id=61a5cd521b194aab81a0ff7939ebd317, user_id=4b3ba696b60e4dc9b69780cc894e58a8, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:07.472288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:07.553113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fad941-44b0-4566-ac24-d2f88ece830d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:07.590345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6746c3-71eb-4777-8946-6a92c57646ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:07.591239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6746c3-71eb-4777-8946-6a92c57646ad None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.591717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6746c3-71eb-4777-8946-6a92c57646ad None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.592129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6746c3-71eb-4777-8946-6a92c57646ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.607469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 65/132] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:07 2022] PUT /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/4b3ba696b60e4dc9b69780cc894e58a8/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:07.608327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.685722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:07.716178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6746c3-71eb-4777-8946-6a92c57646ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:07.720446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6746c3-71eb-4777-8946-6a92c57646ad None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:07.751024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:07.799217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29cc4-7018-48f8-92e7-b8caaa38bdbe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:07.857347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64482034-cdb3-4e4b-9d38-ef02a37278de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:07.857963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64482034-cdb3-4e4b-9d38-ef02a37278de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.858254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64482034-cdb3-4e4b-9d38-ef02a37278de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.858537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64482034-cdb3-4e4b-9d38-ef02a37278de None admin] PATH_INFO: `/v3/projects/84f6e66f87dc45f58cba121ce862b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.859750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64482034-cdb3-4e4b-9d38-ef02a37278de None admin] RBAC: Authorizing `identity:get_project(project_id=84f6e66f87dc45f58cba121ce862b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:07.861267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64482034-cdb3-4e4b-9d38-ef02a37278de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:07.863158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 66/133] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:07 2022] GET /identity/v3/projects/84f6e66f87dc45f58cba121ce862b0f5 => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:07.864190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25176894-4137-436a-b395-a713af7682c0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.864541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25176894-4137-436a-b395-a713af7682c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.864854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25176894-4137-436a-b395-a713af7682c0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.866370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 67/134] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:07.870287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25176894-4137-436a-b395-a713af7682c0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:07.895057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 68/135] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:07.896594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:07.897618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:07.898042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:07.919105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:07.921096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:08.023124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25176894-4137-436a-b395-a713af7682c0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:08.042375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 69/136] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:08.043238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.090010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:08.098679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:08.099575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.099864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.100148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] PATH_INFO: `/v3/projects/84f6e66f87dc45f58cba121ce862b0f5/users/4b3ba696b60e4dc9b69780cc894e58a8/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.102290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] RBAC: Authorizing `identity:create_grant(project_id=84f6e66f87dc45f58cba121ce862b0f5, user_id=4b3ba696b60e4dc9b69780cc894e58a8, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:08.107821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:08.108154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:08.189415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff2cee-2950-4e99-9917-d3a3260c8a76 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:08.201011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 68/137] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:07 2022] PUT /identity/v3/projects/84f6e66f87dc45f58cba121ce862b0f5/users/4b3ba696b60e4dc9b69780cc894e58a8/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:22:08.202306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.227203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:08.228848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.229306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.229729 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.231100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:08.232614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:08.234633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 70/138] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:08.241430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.341286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:08.355171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bbfd9-da26-47cc-b8e5-071e2e8c9aff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:08.395720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:08.396541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.396956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.397343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.398422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] RBAC: Adding query filter params (role.id=a0cbbd58db464ac994fa37db8c9b9fdd, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=7f086f0d540c4d88934e311e9d38adcd) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:08.398844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:08.401438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:08.412092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 71/139] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/role_assignments?user.id=7f086f0d540c4d88934e311e9d38adcd&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=a0cbbd58db464ac994fa37db8c9b9fdd => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:08.468485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393084f9-3a39-46b7-8fd3-6428c3946fdc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:08.520166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:08.521066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.521359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.521645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.538650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:08.547673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:08.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:08.549357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 69/140] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:08.555057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.681297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba7d64-d742-4674-aab2-b9373282e38f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:08.729866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:08.730569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.730857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.731136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.731917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:08.732229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:08.734077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:08.743164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 72/141] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:08.749461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.815642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:08.862822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c51674-65ff-484e-9281-26ce7bd232ed None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:08.897248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:08.898138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.898433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.898919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.900149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:08.902545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:08.904336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 70/142] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:08.913765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.931067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:09.053067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680b45b-2c31-4e5c-ab86-7bc0d34dbfd9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:09.100500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c44ca1-9e62-464b-ab81-ea57dc13b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:09.101191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c44ca1-9e62-464b-ab81-ea57dc13b54a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.101606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c44ca1-9e62-464b-ab81-ea57dc13b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.101918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c44ca1-9e62-464b-ab81-ea57dc13b54a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.190760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c44ca1-9e62-464b-ab81-ea57dc13b54a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:09.192244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c44ca1-9e62-464b-ab81-ea57dc13b54a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:09.216063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:09.342328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 73/143] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:09.348249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:09.414890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e5a4c-f97c-482b-a40a-76f22d585c36 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.415386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e5a4c-f97c-482b-a40a-76f22d585c36 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.415601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e5a4c-f97c-482b-a40a-76f22d585c36 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.417893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 74/144] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:09.423200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.423487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.423771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.452877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:09.454090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:09.532003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa2b77-5f73-49ea-a4d8-e5f3704ae0c8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:09.573830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:09.586871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:09.588402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.588819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.589125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.590465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:09.591937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:09.594224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 71/145] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:09 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:09.599888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:09.612459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 75/146] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:09.648644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:09.717797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e4eb5-f38c-41f0-ba71-3c1861cd8528 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:09.760999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:09.761448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:09.762150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.762436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.762736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.763693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] RBAC: Adding query filter params (role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=4b3ba696b60e4dc9b69780cc894e58a8) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:09.764070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:09.765911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0973c-6524-4995-949a-eca25166769c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:09.772931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 72/147] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:09 2022] GET /identity/v3/role_assignments?user.id=4b3ba696b60e4dc9b69780cc894e58a8&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:09.803809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1750c5-434d-4b6c-9f59-7defa6d03fe4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.804131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1750c5-434d-4b6c-9f59-7defa6d03fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.804406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1750c5-434d-4b6c-9f59-7defa6d03fe4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.806106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 73/148] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:09.813141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.813495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.813784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.847216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:09.848351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:09.885361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24af203-2cb0-40c6-89c8-f46375610eee None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:09.961128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:09.962412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.962756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.963068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.964394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:09.966088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:09.967992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 76/149] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:09.973960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:09.995572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.051098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 74/150] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:10.093096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5891-dbff-4ce9-9062-2df6003b57fe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:10.097850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:10.107153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.137160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:10.138066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.138380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.138685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.139784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:10.141325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:10.244825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c247d-beea-43b0-8954-a132ef7ee12a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:10.260486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 77/151] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:09 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:22:10.306530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:10.307629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.307972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.308301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.323948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:10.331946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:10.341735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:10.342496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 75/152] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:10.343160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:10.473752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e48f9e-97e5-45bb-a3fe-a972453c7c11 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:10.515164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:10.515886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.516170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.516495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.517363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:10.517720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:10.519120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:10.534232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 78/153] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:10.539971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:10.593565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.658198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731a055-9dde-4f2c-b31c-18d41ae3c2ec None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:10.693914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915712f4-b79e-44a1-9af4-3f6fc558cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:10.694793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915712f4-b79e-44a1-9af4-3f6fc558cdae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.695078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915712f4-b79e-44a1-9af4-3f6fc558cdae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.695357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915712f4-b79e-44a1-9af4-3f6fc558cdae None admin] PATH_INFO: `/v3/users/02055864266e49b39ebadd610e2d403c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.696553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915712f4-b79e-44a1-9af4-3f6fc558cdae None admin] RBAC: Authorizing `identity:get_user(user_id=02055864266e49b39ebadd610e2d403c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:10.698713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915712f4-b79e-44a1-9af4-3f6fc558cdae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:10.700341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 76/154] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:10 2022] GET /identity/v3/users/02055864266e49b39ebadd610e2d403c => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:10.706077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:10.740299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.756055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8d579-390f-44e3-a1bc-50a9fb64e630 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.756369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8d579-390f-44e3-a1bc-50a9fb64e630 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.756611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8d579-390f-44e3-a1bc-50a9fb64e630 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.758155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 77/155] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:10.766922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.766922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.766922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.794674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:10.796601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:10.836947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33d501-5e50-4d98-be76-44219e6c063f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:10.884522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069b377-af2a-4115-ac9a-6973d88b620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:10.885798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069b377-af2a-4115-ac9a-6973d88b620e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.886085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069b377-af2a-4115-ac9a-6973d88b620e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.886368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069b377-af2a-4115-ac9a-6973d88b620e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.889596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 78/156] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:10.913202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:10.960636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.968204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069b377-af2a-4115-ac9a-6973d88b620e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:10.969713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069b377-af2a-4115-ac9a-6973d88b620e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:11.000763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.026383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70890-634d-4db8-9fd3-6befd0b2c2fd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.065843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.066789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.067197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.067574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.081518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:11.082986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 79/157] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:11.084499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0ca8c-b4ce-4fa1-8eee-877de20b63bb None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.084866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0ca8c-b4ce-4fa1-8eee-877de20b63bb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.085146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0ca8c-b4ce-4fa1-8eee-877de20b63bb None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.086408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 80/158] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:11.091879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.092166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.092442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:11.094834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:11.095655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:11.096489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 79/159] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:11.101926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.112597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:11.114292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:11.203190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 81/160] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:11.204320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.228792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7b5c-d296-46c7-b9f0-fb15eac7b3b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.228792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.264638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.265699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.266267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.266685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.267727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:11.268251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:11.270695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:11.280167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 80/161] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:11.281401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.310153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.370155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c9ae-9ae5-4483-9fa2-433c16d1734f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.404532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790dd5d-9713-4aa7-b669-2f1aa43b769e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.425696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.427406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.427993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.428562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.430289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:11.434072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:11.436707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 82/162] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:11.437818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.454808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.455691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.456043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.456404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.473668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:11.483677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:11.484558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:11.485433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 81/163] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:11.486054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.524190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.577588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a838a3b-0a02-4302-8a9a-a3b7adb6a016 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.614657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.622557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042488a6-7aed-44c3-8d84-576077f4908a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.627722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.630699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.631053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.631402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.632790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:11.634330 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:11.637369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 83/164] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:11.638241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.667021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.668309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.668813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.669447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.671071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=02055864266e49b39ebadd610e2d403c) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:11.671572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:11.674010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:11.682527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 82/165] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/role_assignments?user.id=02055864266e49b39ebadd610e2d403c&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:11.683637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.793176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.794736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d032da-ca33-4f4f-a704-e3dd779fb2d2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.805550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aba82-9f5e-4f56-8605-43a475f1c046 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:11.844209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd0f04-1533-45c4-a7d4-c625db1b9e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.845368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd0f04-1533-45c4-a7d4-c625db1b9e7d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.845823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd0f04-1533-45c4-a7d4-c625db1b9e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.846260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd0f04-1533-45c4-a7d4-c625db1b9e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.859454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:11.860516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.860655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.861075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.861920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:11.862293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:11.863849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc55d4c-0f58-462c-a100-c8ec3b62416a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:11.884552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 84/166] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:11.886647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00504b5-ee1f-4fa2-8596-2868dc2992d6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.886647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00504b5-ee1f-4fa2-8596-2868dc2992d6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.886647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00504b5-ee1f-4fa2-8596-2868dc2992d6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.887873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 85/167] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:11.893515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00504b5-ee1f-4fa2-8596-2868dc2992d6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:11.925460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd0f04-1533-45c4-a7d4-c625db1b9e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:11.927412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd0f04-1533-45c4-a7d4-c625db1b9e7d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.018856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.044243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.062881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 83/168] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:12.065126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.065126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.065419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.080207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00504b5-ee1f-4fa2-8596-2868dc2992d6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.086348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:12.087563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:12.120263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.121612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.122102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.122581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] PATH_INFO: `/v3/users/7f086f0d540c4d88934e311e9d38adcd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.124056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] RBAC: Authorizing `identity:get_user(user_id=7f086f0d540c4d88934e311e9d38adcd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.126420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.130401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 86/169] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:11 2022] GET /identity/v3/users/7f086f0d540c4d88934e311e9d38adcd => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:12.131409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.173042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 84/170] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:12.173922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.247344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165cdec-9520-470c-9e84-0e1e2dfb87e9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.280410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.292139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.293330 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.294759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.294759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.295622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.297333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.299170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 87/171] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:12.300054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.315101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fcc2f-f34b-43a5-90a1-1b2c36395016 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.336294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.356604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.357806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.358234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.358698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.360088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.361891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.364594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 85/172] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:12.365592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.479747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece6c4d-4349-4e18-8397-1688a28bfcd9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.519255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.520667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.521197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.521649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.522794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=4b3ba696b60e4dc9b69780cc894e58a8) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:12.523292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.525700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.534240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 86/173] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/role_assignments?user.id=4b3ba696b60e4dc9b69780cc894e58a8&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=19944ab23542462988b68290167ac622 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:12.535587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.578825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9d4df-8408-4bdf-b283-1607392daaa1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.600719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.658719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0376-863d-4ad8-ab7e-425f493a41d6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.698902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.700015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.700419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.700836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/7f086f0d540c4d88934e311e9d38adcd/roles/a0cbbd58db464ac994fa37db8c9b9fdd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.703232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=61a5cd521b194aab81a0ff7939ebd317, user_id=7f086f0d540c4d88934e311e9d38adcd, role_id=a0cbbd58db464ac994fa37db8c9b9fdd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.704849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.729488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.730374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.730669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.730956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.749047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:12.757232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:12.758100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:12.758873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 88/174] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:12.764789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.783344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 87/175] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:12 2022] PUT /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317/users/7f086f0d540c4d88934e311e9d38adcd/roles/a0cbbd58db464ac994fa37db8c9b9fdd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:12.830507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.883548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a7465-3632-4582-9eb9-ca410f23e28a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.921083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.921766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.922054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.922335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.923105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:12.923417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.924832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.934713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 89/176] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:12.941847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:13.081320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762cd36-8e5d-483a-b369-dea0e912d3d2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:13.096812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:13.124466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:13.125385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.125668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.125947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] PATH_INFO: `/v3/users/4b3ba696b60e4dc9b69780cc894e58a8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.127180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3ba696b60e4dc9b69780cc894e58a8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:13.128633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:13.130502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 88/177] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/users/4b3ba696b60e4dc9b69780cc894e58a8 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:13.136913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:13.271558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5a3b7-f6d6-4be5-a209-f8b4327c1e03 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:13.293777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:13.325255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:13.325927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.326211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.326489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.410090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:13.411512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:13.545730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 90/178] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:13.551473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:13.714229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b696416-571d-46da-a2d9-f099f5c406b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:13.761089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31121ad1-6d30-4aaa-8735-6769c9a39a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:13.761959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31121ad1-6d30-4aaa-8735-6769c9a39a95 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.762245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31121ad1-6d30-4aaa-8735-6769c9a39a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.762526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31121ad1-6d30-4aaa-8735-6769c9a39a95 None admin] PATH_INFO: `/v3/projects/84f6e66f87dc45f58cba121ce862b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.763719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31121ad1-6d30-4aaa-8735-6769c9a39a95 None admin] RBAC: Authorizing `identity:get_project(project_id=84f6e66f87dc45f58cba121ce862b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:13.765220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31121ad1-6d30-4aaa-8735-6769c9a39a95 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:13.767094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 89/179] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:13 2022] GET /identity/v3/projects/84f6e66f87dc45f58cba121ce862b0f5 => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:13.773126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:13.809843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:13.889780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15921dcb-c128-4df1-90c0-66871998c774 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.890100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15921dcb-c128-4df1-90c0-66871998c774 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.890351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15921dcb-c128-4df1-90c0-66871998c774 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.891990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 90/180] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:13.896812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1b9ee-ec52-47f7-a822-b6842fe2b2ce None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.897116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1b9ee-ec52-47f7-a822-b6842fe2b2ce None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.897389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1b9ee-ec52-47f7-a822-b6842fe2b2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.914062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9cb1b9ee-ec52-47f7-a822-b6842fe2b2ce None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:13.915259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9cb1b9ee-ec52-47f7-a822-b6842fe2b2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:13.948256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24612-c1be-4d7c-960a-032191b205b6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:13.980004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.012518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.013275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.013478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.013751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.014603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, scope.project.id=84f6e66f87dc45f58cba121ce862b0f5, user.id=4b3ba696b60e4dc9b69780cc894e58a8) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:14.014918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:14.016359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:14.031343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 91/182] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:13 2022] GET /identity/v3/role_assignments?user.id=4b3ba696b60e4dc9b69780cc894e58a8&scope.project.id=84f6e66f87dc45f58cba121ce862b0f5&role.id=19944ab23542462988b68290167ac622 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:14.036947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 91/182] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:14.054442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.174249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.180943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841177-80e0-465b-ae06-66a1982ecae2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.181226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841177-80e0-465b-ae06-66a1982ecae2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.190833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841177-80e0-465b-ae06-66a1982ecae2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.190833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 92/183] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:14.190833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.190833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.190833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.209187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b66562-98fb-450d-98dd-3b4b3fd0ad38 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.229775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:14.231053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:14.255670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.256760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.257204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.257618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.269400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:14.278259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:14.279130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:14.279910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:14.283297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 92/184] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:14.293688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.331547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 93/185] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:14.359092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.410452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad202-2243-4246-a4ac-a490a3193ff8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.449255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.449920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.450202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.450481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.451237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:14.451544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:14.452973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c245ed7-b4b0-4c0f-acb8-552a21572eeb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:14.459016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.463299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 93/186] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:14.464582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cf741-549e-4b72-a01a-0ec10b8475b0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.465195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cf741-549e-4b72-a01a-0ec10b8475b0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.465479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cf741-549e-4b72-a01a-0ec10b8475b0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.467103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 94/187] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:14.471528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cf741-549e-4b72-a01a-0ec10b8475b0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.478903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4153c35-e45c-4acc-a2e1-088c394d542c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.487249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.513746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1b9dc-3616-4d80-acd2-ce3b456e9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.514562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1b9dc-3616-4d80-acd2-ce3b456e9320 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.514853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1b9dc-3616-4d80-acd2-ce3b456e9320 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.515135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1b9dc-3616-4d80-acd2-ce3b456e9320 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.525434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1b9dc-3616-4d80-acd2-ce3b456e9320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4ae1b9dc-3616-4d80-acd2-ce3b456e9320 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:14.533947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:14.534839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 94/188] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:14.539961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.541294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.541294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.558976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:14.560160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:14.586044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cf741-549e-4b72-a01a-0ec10b8475b0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.624395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.625333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.625635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.625930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] PATH_INFO: `/v3/users/f358d57309cb45c881901c818e27429b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.629980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 95/189] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:14.630721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.643200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] RBAC: Authorizing `identity:get_user(user_id=f358d57309cb45c881901c818e27429b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:14.645019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:14.647530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 95/190] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/users/f358d57309cb45c881901c818e27429b => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:14.653589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.715755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.745969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.751709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b68ffb-e22a-4d24-921a-784de0200669 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.758294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0da7f-87d1-43b5-8195-8b38428ab8c5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.790505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.791755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.792051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.792304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.793086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:14.793392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:14.794762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:14.794987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.795642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.796034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.796208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.804012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 96/191] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:14.805914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.876467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:14.877981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:14.913757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed73e7b-9018-4071-b5fa-403840f95cf1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:14.937384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.949858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:14.953039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.953039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.953039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.953039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:14.953864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:14.960382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 97/192] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:14.960382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:14.985392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:14.998487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 96/193] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:14.998487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.061135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa59b2-000c-4846-981c-acb681487ea7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.099523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.100409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.100721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.101013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] PATH_INFO: `/v3/users/f358d57309cb45c881901c818e27429b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.102202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] RBAC: Authorizing `identity:get_user(user_id=f358d57309cb45c881901c818e27429b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.103850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.105566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 98/194] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:14 2022] GET /identity/v3/users/f358d57309cb45c881901c818e27429b => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:15.110188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.110264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa65ec-9ebc-49ca-8963-eab4d5a46ef9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.145290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.145971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.146277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.146537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.147266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.148728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.206084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.213713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3663-3d72-479b-a94d-caf3d01a7e63 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.237139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 97/195] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:14 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:15.249953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.254488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.255727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.257082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.257759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.267613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.270414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.273137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 99/196] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:15.275749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.279618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.379246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe526cb-d2ba-4cd6-8136-9d27c67591a5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.401354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114803-6c71-489e-83fc-9d7a6a9e2515 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.421827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.422490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.422770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.423101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.441128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.441797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.442156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.442516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.443355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=f358d57309cb45c881901c818e27429b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:15.443696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.445175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51157c2-e058-4b75-b308-f730c648b99e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.453708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 100/197] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/role_assignments?user.id=f358d57309cb45c881901c818e27429b&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:15.462287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347a181-cac0-4587-88f6-198df3bf43b8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.462682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347a181-cac0-4587-88f6-198df3bf43b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.464863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347a181-cac0-4587-88f6-198df3bf43b8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.464863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 101/198] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:15.474223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.474514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.474800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.498527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:15.499893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:15.507022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.507022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.528068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.530368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.604612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 102/199] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:15.628557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.667658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 98/200] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:15.673642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.789052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc544e7-a123-4bc3-8d34-d5a4775ed03d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.797875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc35c5-36b8-4660-b897-abb6ab53ff9b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.826306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.841120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.841930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.842213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.842493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.864197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:15.874125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.875045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.875335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.875613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.877098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.878569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:15.882170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:15.883022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:15.883872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 103/201] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:15.884571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 99/202] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:15.886947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.891035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.927206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:16.031005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9480-976f-4fcc-81d0-9fcff9fc9644 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:16.077851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce0dd7-05a5-4de1-9346-3c2707d1947a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:16.081878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:16.082784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.083068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.083347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.084190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] RBAC: Adding query filter params (role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=f358d57309cb45c881901c818e27429b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:16.084509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:16.086445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:16.095012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 104/203] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/role_assignments?user.id=f358d57309cb45c881901c818e27429b&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:16.156348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:16.157081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.157369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.157652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.158436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:16.158745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:16.160173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:16.176138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 100/204] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:16.182485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:16.318843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:16.339664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eca7d3-1c00-479b-9417-996ae8ee358c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:16.375392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:16.376323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.376613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.376922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] PATH_INFO: `/v3/users/7f086f0d540c4d88934e311e9d38adcd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.378121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] RBAC: Authorizing `identity:get_user(user_id=7f086f0d540c4d88934e311e9d38adcd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:16.379560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:16.381414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 105/205] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:16 2022] GET /identity/v3/users/7f086f0d540c4d88934e311e9d38adcd => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:16.389826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:16.451720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:16.528426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9086107-802b-48fd-a774-75a18d8e539a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:16.577492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:16.578151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.578439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.578724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.689801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:16.691202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:16.738171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:16.823803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 101/206] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:16.829487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:16.977188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2f775-6d96-4b15-b9bd-b48811360e7e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:17.031304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbe9ca-66dc-4f7e-9926-ea39df07ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:17.032177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe9ca-66dc-4f7e-9926-ea39df07ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.032459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe9ca-66dc-4f7e-9926-ea39df07ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.032762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe9ca-66dc-4f7e-9926-ea39df07ade2 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.033968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbe9ca-66dc-4f7e-9926-ea39df07ade2 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:17.035421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbe9ca-66dc-4f7e-9926-ea39df07ade2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:17.037457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 106/207] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:16 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:17.043826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:17.199986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175cca8-c3e4-4949-ac36-b14ab5cd1ff0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:17.234652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:17.235232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.235521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.235799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.236635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] RBAC: Adding query filter params (role.id=a0cbbd58db464ac994fa37db8c9b9fdd, scope.project.id=61a5cd521b194aab81a0ff7939ebd317, user.id=7f086f0d540c4d88934e311e9d38adcd) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:17.236981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:17.238412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac39ac-ca56-4664-9262-9524c38d82f3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:17.244941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 102/208] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:17 2022] GET /identity/v3/role_assignments?user.id=7f086f0d540c4d88934e311e9d38adcd&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=a0cbbd58db464ac994fa37db8c9b9fdd => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:17.849479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95365d-2374-4204-9787-5a5da38ed7df None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.850015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95365d-2374-4204-9787-5a5da38ed7df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.850015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95365d-2374-4204-9787-5a5da38ed7df None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.851541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 107/209] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:17.858433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638b887-1db7-4d3a-89e8-33bc9f3dad1f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.858980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638b887-1db7-4d3a-89e8-33bc9f3dad1f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.859528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638b887-1db7-4d3a-89e8-33bc9f3dad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.878841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9638b887-1db7-4d3a-89e8-33bc9f3dad1f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:17.880346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9638b887-1db7-4d3a-89e8-33bc9f3dad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:17.882637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:17.953998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 103/210] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:17.970124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9f50-56c8-4fc5-81dd-dbfb928b6718 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.970417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9f50-56c8-4fc5-81dd-dbfb928b6718 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.970707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9f50-56c8-4fc5-81dd-dbfb928b6718 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.972255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 108/211] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:17.975200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbf517-d2ba-4442-af82-36f5f26a06ce None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.975495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbf517-d2ba-4442-af82-36f5f26a06ce None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.975778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbf517-d2ba-4442-af82-36f5f26a06ce None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.977352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 104/212] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:17.977458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a5e5-757a-4512-a75e-50a7325299a0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.977681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a5e5-757a-4512-a75e-50a7325299a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.977969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a5e5-757a-4512-a75e-50a7325299a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.980603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbf517-d2ba-4442-af82-36f5f26a06ce None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:17.997588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5b43a5e5-757a-4512-a75e-50a7325299a0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:17.998780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b43a5e5-757a-4512-a75e-50a7325299a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:18.011138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.069240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 109/213] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:18.070802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.071293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.071736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.092716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbf517-d2ba-4442-af82-36f5f26a06ce None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.101726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:18.103288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:18.129866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.130799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.131088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.131364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.139169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.143337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:18.152144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:18.154146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 105/214] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:18.155217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.179584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 110/215] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:18.180260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.269289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.282712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33be2-cd8a-49ec-985e-4233cd28cfce None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.293898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf69ac0-e87f-4c39-ae75-5beeb018f47b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.319403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.320204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.320493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.320804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.330627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.331296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.331583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.331756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:18.331851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.332974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:18.333202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.334574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:18.341455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:18.349074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 106/216] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:18.349883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.350590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 111/217] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:18.350590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.411683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.455412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa477d-7025-48ac-8979-65d39edb64b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.467298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633699c-a600-4f73-b57c-8c756d9b4696 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.496275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.497102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.497390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.497670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.498859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.500343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.501971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 107/218] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:18.502749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.507441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.508326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.508610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.509115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] PATH_INFO: `/v3/users/f358d57309cb45c881901c818e27429b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.511269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] RBAC: Authorizing `identity:get_user(user_id=f358d57309cb45c881901c818e27429b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.512736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.514513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 112/219] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/users/f358d57309cb45c881901c818e27429b => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:18.517514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.528439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.615233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af075c-48a7-434a-b577-4c6c357d0aa2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.622610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2285a6-279e-4e9f-a85f-b3c9bd5abbfb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.639560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.647737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.648441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.648755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.649036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.656822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:18.656822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.656822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.657087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31849e57-163c-4937-8ac0-907388a1bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.657087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31849e57-163c-4937-8ac0-907388a1bf07 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.657225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31849e57-163c-4937-8ac0-907388a1bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.657417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31849e57-163c-4937-8ac0-907388a1bf07 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.658119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31849e57-163c-4937-8ac0-907388a1bf07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.659542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31849e57-163c-4937-8ac0-907388a1bf07 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.660524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 108/220] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:18.661349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.754939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 113/221] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:18 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:18.756255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1aa59-9156-484b-9607-c6d73abdf8d7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.756569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1aa59-9156-484b-9607-c6d73abdf8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.756887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1aa59-9156-484b-9607-c6d73abdf8d7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.760863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 114/222] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:18.760863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1aa59-9156-484b-9607-c6d73abdf8d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.774866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a079174-aff7-4a0d-a99f-98bd4c8f5ac3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.816191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0fe7b-a332-420f-9526-4f01d9ff65ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.817102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0fe7b-a332-420f-9526-4f01d9ff65ac None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.817385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0fe7b-a332-420f-9526-4f01d9ff65ac None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.817659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0fe7b-a332-420f-9526-4f01d9ff65ac None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.818892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0fe7b-a332-420f-9526-4f01d9ff65ac None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.820351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0fe7b-a332-420f-9526-4f01d9ff65ac None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.823417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 109/223] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:18.824608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.824917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.825201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.844048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:18.845288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:18.851252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.906598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1aa59-9156-484b-9607-c6d73abdf8d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.907564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.927668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 110/224] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:18.928788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.949964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.950683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.951061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.951790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] PATH_INFO: `/v3/users/e8d821ab686f49b3b9a3decc1d1cc389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.968572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] RBAC: Authorizing `identity:get_user(user_id=e8d821ab686f49b3b9a3decc1d1cc389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.970090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.971991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 115/225] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/users/e8d821ab686f49b3b9a3decc1d1cc389 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:18.972585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.064652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e72e2c-4efd-4bda-b9ec-4503ed22dcb2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.106279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b197ad2-571f-4e4e-8199-c21c273cde4d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.121543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.122057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.122057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.122238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/f358d57309cb45c881901c818e27429b/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.125651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] RBAC: Authorizing `identity:create_grant(project_id=7d83853bf6704e0fa8f738c857c951c2, user_id=f358d57309cb45c881901c818e27429b, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.128411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.135727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.154925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.156033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.156527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.157016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.169558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:19.179755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:19.180936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 116/226] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:19.185691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.213324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 111/227] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:18 2022] PUT /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/f358d57309cb45c881901c818e27429b/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:19.214719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.277814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.344617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a943-e5e1-4654-94f8-a74c1599c1aa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.389167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.389990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.390392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.390766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.391645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:19.392509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.393696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.404938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 112/228] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:19.407964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e62ae0-11a0-47ca-ae7d-35cd8362abcd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.411276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.449284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.450467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.451081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.451558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.506762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.527072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c22e1-9c33-4194-a4c4-da8762ff6142 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.529670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.531369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.536307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.564000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.564950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.565405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.565585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] PATH_INFO: `/v3/users/f358d57309cb45c881901c818e27429b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.566757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] RBAC: Authorizing `identity:get_user(user_id=f358d57309cb45c881901c818e27429b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.568286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.570595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 113/229] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/users/f358d57309cb45c881901c818e27429b => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:19.576494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.643347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 117/230] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:19.649031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.686261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca7d65-807f-4825-8c15-b69a51524184 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.723001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.723866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.724147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.724425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.726500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.727298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.730234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 114/231] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:19.735775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.739131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.761562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717842f5-442d-4276-8743-741a96ec76d4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.783722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.800224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.801434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.801940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.802415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.803794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.805466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.807844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 118/232] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:19.817754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.848893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725c38e-62c5-488c-a476-b871fafe6854 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.918701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02b30d-c093-42e3-b02d-7611996d2aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.919582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02b30d-c093-42e3-b02d-7611996d2aa9 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.919869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02b30d-c093-42e3-b02d-7611996d2aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.920147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02b30d-c093-42e3-b02d-7611996d2aa9 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/f358d57309cb45c881901c818e27429b/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.926558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02b30d-c093-42e3-b02d-7611996d2aa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d83853bf6704e0fa8f738c857c951c2, user_id=f358d57309cb45c881901c818e27429b, role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.928007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02b30d-c093-42e3-b02d-7611996d2aa9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.932033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b606-9572-46d5-8c91-1c2c7ea97965 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.970276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.971308 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.971838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.972356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.973500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=e8d821ab686f49b3b9a3decc1d1cc389) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:19.974031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:19.975739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06554088-ad67-4b02-89ea-158c721d5008 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:19.983685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 119/233] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/role_assignments?user.id=e8d821ab686f49b3b9a3decc1d1cc389&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=19944ab23542462988b68290167ac622 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:20.050046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 115/234] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:19 2022] PUT /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/f358d57309cb45c881901c818e27429b/roles/6413d71218d344e8b26c8577cac9c4dc => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:21.287276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602be31-e10f-4627-9454-cbce02fc9e47 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.287785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602be31-e10f-4627-9454-cbce02fc9e47 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.287860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602be31-e10f-4627-9454-cbce02fc9e47 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.289533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 120/235] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:21.295243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0629268-3ecc-4224-8dd6-197251e85146 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.295655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0629268-3ecc-4224-8dd6-197251e85146 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.295887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0629268-3ecc-4224-8dd6-197251e85146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.321723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c0629268-3ecc-4224-8dd6-197251e85146 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:21.323018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c0629268-3ecc-4224-8dd6-197251e85146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:21.409614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 116/236] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:21.437139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602be31-e10f-4627-9454-cbce02fc9e47 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:21.438449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f478c1a-0740-4071-9079-db4aa154ec3c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.438752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f478c1a-0740-4071-9079-db4aa154ec3c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.439042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f478c1a-0740-4071-9079-db4aa154ec3c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.440855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 117/237] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:21.450358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.450650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.450939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.467282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:21.479966 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:21.490569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:21.521159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:21.595232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 118/238] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:21.615644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602be31-e10f-4627-9454-cbce02fc9e47 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:21.619617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:21.660341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:21.661179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.661466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.661747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.682961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:21.692480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:21.693373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:21.694170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 121/239] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:21.701688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:21.751687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7069c-c4dd-4609-b862-abb604932130 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:21.789795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b086-e2bd-47d9-9c2c-1fda4ffb54c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:21.791067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b086-e2bd-47d9-9c2c-1fda4ffb54c3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.791536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b086-e2bd-47d9-9c2c-1fda4ffb54c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.791819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b086-e2bd-47d9-9c2c-1fda4ffb54c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.793038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897b086-e2bd-47d9-9c2c-1fda4ffb54c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:21.794500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897b086-e2bd-47d9-9c2c-1fda4ffb54c3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:21.800718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 119/240] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:21.802015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c31cc-79b7-4181-8e86-3a39aff6625f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.802311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c31cc-79b7-4181-8e86-3a39aff6625f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.802594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c31cc-79b7-4181-8e86-3a39aff6625f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.803907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 120/241] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:21.809097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c31cc-79b7-4181-8e86-3a39aff6625f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:21.809523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:21.844152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:21.852444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d54500-99c4-4e37-bc42-0dceca492ed9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:21.894660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:21.895628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.896095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.896553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.897652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:21.898184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:21.899940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc453a-2bd4-46fe-afcf-8b29016be73e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:21.909319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 122/242] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:21.915323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.916438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.916947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.923779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c31cc-79b7-4181-8e86-3a39aff6625f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:21.934752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:21.936225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:21.959784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:21.960460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.960797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.961337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.962070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:21.964343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:22.010121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 123/243] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:22.011278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.065073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 121/244] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:21 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:22.066379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.127878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:22.212050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34da57f-83a9-44e4-bc3d-aedb3d9cf2b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:22.260320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:22.281010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643c0aa-4894-458e-99c6-56e246ef9fbb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:22.283852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:22.284882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.285230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.285513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] PATH_INFO: `/v3/users/2c34492ca5a649cb959a470ba3832475` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.305749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] RBAC: Authorizing `identity:get_user(user_id=2c34492ca5a649cb959a470ba3832475)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:22.307349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:22.309138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 124/245] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/users/2c34492ca5a649cb959a470ba3832475 => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:22.315591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.323986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:22.324828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.325119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.326255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.341280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:22.350615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:22.351499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:22.352288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:22.353987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 122/246] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:22.360022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.469160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5fdda-1aa3-4047-bc0d-8ad5737155c7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:22.491126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bd071-7db5-496e-97bd-7021d5cd2153 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:22.500433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:22.510521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57c3c5-3d0d-4720-9e71-2375fd8249e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:22.511176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57c3c5-3d0d-4720-9e71-2375fd8249e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.511466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57c3c5-3d0d-4720-9e71-2375fd8249e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.511747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57c3c5-3d0d-4720-9e71-2375fd8249e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.530885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:22.531571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.531855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.532135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.532947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:22.533262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:22.534672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:22.546259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 123/247] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:22.553733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.572596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:22.617037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57c3c5-3d0d-4720-9e71-2375fd8249e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:22.618617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57c3c5-3d0d-4720-9e71-2375fd8249e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:22.704144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcb7c0-84af-446c-8a5e-c7c81a148114 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:22.739187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fac1e-6203-4a45-b74c-ac33937c140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:22.740063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fac1e-6203-4a45-b74c-ac33937c140b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.740352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fac1e-6203-4a45-b74c-ac33937c140b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.740633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fac1e-6203-4a45-b74c-ac33937c140b None admin] PATH_INFO: `/v3/users/f358d57309cb45c881901c818e27429b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.741937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fac1e-6203-4a45-b74c-ac33937c140b None admin] RBAC: Authorizing `identity:get_user(user_id=f358d57309cb45c881901c818e27429b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:22.743389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fac1e-6203-4a45-b74c-ac33937c140b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:22.745406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 125/248] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:22:22.749588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 124/249] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/users/f358d57309cb45c881901c818e27429b => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:22.749588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e71994-7759-45fc-9897-b7334372739a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.749588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e71994-7759-45fc-9897-b7334372739a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.749588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e71994-7759-45fc-9897-b7334372739a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.749588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 125/250] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:22.751217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b33cf-755b-4a15-b3ee-29736bb5296f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.751519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b33cf-755b-4a15-b3ee-29736bb5296f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.751800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b33cf-755b-4a15-b3ee-29736bb5296f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.757418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 126/251] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:22.757543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e71994-7759-45fc-9897-b7334372739a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.757620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f21d7-a329-43d1-b8c1-a0907d0abafd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.757620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f21d7-a329-43d1-b8c1-a0907d0abafd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.757620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f21d7-a329-43d1-b8c1-a0907d0abafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.773784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-910f21d7-a329-43d1-b8c1-a0907d0abafd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:22.775949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-910f21d7-a329-43d1-b8c1-a0907d0abafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:22.813150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:22.842824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:22.864450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 127/252] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:22.865901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.866263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.866556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.874476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e71994-7759-45fc-9897-b7334372739a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:22.898579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:22.900139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:22.920158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:22.921476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:22.921868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:22.922651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:22.924002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:22.926177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:22.929239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 126/253] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:22.930500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:22.990562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 128/254] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:22.991534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.070237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c83dc-93df-492e-a807-0c7df219a0b6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.078782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.098057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65204473-bb36-4d26-be5a-58486a04a902 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.108852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.109758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.110226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.110684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.128008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.133417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.134393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.134687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.135027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.149653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:23.159770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:23.160647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:23.161489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:23.164453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 129/255] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:23.165589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.191809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:23.193802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:23.293163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70a0e-ee83-4dac-8e0a-ca4d4f48527e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.311666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 127/256] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:23.317393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.334024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.334685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.334975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.335251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.336094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] RBAC: Adding query filter params (role.id=a0cbbd58db464ac994fa37db8c9b9fdd, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=2c34492ca5a649cb959a470ba3832475) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:23.336411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:23.337990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:23.348868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 130/257] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/role_assignments?user.id=2c34492ca5a649cb959a470ba3832475&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=a0cbbd58db464ac994fa37db8c9b9fdd => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:23.348868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.376495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.465335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.483090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd6890-9942-4557-99b4-3990a2f0cc59 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.524303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.525057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.525346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.525622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.526545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:23.526858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:23.528264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:23.547169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 131/258] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:23.548034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.572310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec08901-1f76-42c4-8f15-51bcd1889d02 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.614547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.615332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.615619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.615947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.628612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:23.637930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:23.638814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:23.640275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 128/259] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:23.641230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.645073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.716108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f0241-4fd1-49d3-a022-9b358598d9fe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.784058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4163-8be6-4ffa-adef-bae37f58d818 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.788313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.789233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.789520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.789799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.791645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:23.793142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:23.794891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 132/260] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:23.795669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.809049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.856099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.857011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.857302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.857582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] PATH_INFO: `/v3/users/f358d57309cb45c881901c818e27429b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.865095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] RBAC: Authorizing `identity:get_user(user_id=f358d57309cb45c881901c818e27429b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:23.866517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:23.868192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 129/261] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/users/f358d57309cb45c881901c818e27429b => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:23.869520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.997229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72026a5a-a7f9-4b2d-b912-c1f4ef5c22b8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.059290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c88de0-70c9-408c-b465-7985609c08ee None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.077404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:24.078099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.078398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.078684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.079452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:24.079763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:24.081173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:24.090105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:24.100503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:24.106995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.106995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.106995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.106995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=f358d57309cb45c881901c818e27429b) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:24.106995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:24.106995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:24.109187 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 133/262] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:24.110047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:24.111868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 130/263] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/role_assignments?user.id=f358d57309cb45c881901c818e27429b&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:24.118398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:24.185055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:24.301730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636bf96-cbff-4f6d-932e-8c26949f89f2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.379736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:24.380547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.380862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.381146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] PATH_INFO: `/v3/users/e8d821ab686f49b3b9a3decc1d1cc389` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.382674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d821ab686f49b3b9a3decc1d1cc389)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:24.384472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:24.386580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 131/264] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:24 2022] GET /identity/v3/users/e8d821ab686f49b3b9a3decc1d1cc389 => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:24.397389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:24.456347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08300-5078-4036-9c66-365bcc4c3d28 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.561392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de489c-1451-4def-9fe9-75ce50e70011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:24.562070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de489c-1451-4def-9fe9-75ce50e70011 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.562363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de489c-1451-4def-9fe9-75ce50e70011 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.562644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de489c-1451-4def-9fe9-75ce50e70011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.563237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a664516-7011-469b-a84d-b70ea172c987 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.573044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:24.605998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:24.606882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.607164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.607443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.612726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:24.614201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:24.617894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 132/265] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:24 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:24.622330 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:24.668375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:24.672899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de489c-1451-4def-9fe9-75ce50e70011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:24.675009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de489c-1451-4def-9fe9-75ce50e70011 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:24.826616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d74cfc-14ef-4677-933c-8ccb9fe52e28 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.838737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 134/266] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:24.840448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cad0d-b53a-400c-ba9a-5069abe73b01 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.840982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cad0d-b53a-400c-ba9a-5069abe73b01 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.841463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cad0d-b53a-400c-ba9a-5069abe73b01 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.843561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 135/267] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:24.844652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cad0d-b53a-400c-ba9a-5069abe73b01 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:24.877646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee54e1-5ba4-4797-897f-e22dbcde1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:24.878557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee54e1-5ba4-4797-897f-e22dbcde1609 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.878846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee54e1-5ba4-4797-897f-e22dbcde1609 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.879134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee54e1-5ba4-4797-897f-e22dbcde1609 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/e8d821ab686f49b3b9a3decc1d1cc389/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.882118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee54e1-5ba4-4797-897f-e22dbcde1609 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d83853bf6704e0fa8f738c857c951c2, user_id=e8d821ab686f49b3b9a3decc1d1cc389, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:24.883632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee54e1-5ba4-4797-897f-e22dbcde1609 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:24.955986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 133/268] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:24 2022] PUT /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/e8d821ab686f49b3b9a3decc1d1cc389/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:22:24.957832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f245c2-2da2-409b-92e5-935e3c0d85c1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:24.958204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f245c2-2da2-409b-92e5-935e3c0d85c1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:24.958572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f245c2-2da2-409b-92e5-935e3c0d85c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:24.972313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:24.976839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-87f245c2-2da2-409b-92e5-935e3c0d85c1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:24.977576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cad0d-b53a-400c-ba9a-5069abe73b01 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:24.978326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-87f245c2-2da2-409b-92e5-935e3c0d85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:25.011541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:25.021566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231a411-0a51-4fbb-8f07-071094e74d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.022685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231a411-0a51-4fbb-8f07-071094e74d1d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.023948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231a411-0a51-4fbb-8f07-071094e74d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.024244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231a411-0a51-4fbb-8f07-071094e74d1d None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.026599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231a411-0a51-4fbb-8f07-071094e74d1d None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:25.028232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231a411-0a51-4fbb-8f07-071094e74d1d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:25.031012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 136/269] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:24 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:25.032666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506a466-fbb1-4708-bf90-97dc8a54d503 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.032991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506a466-fbb1-4708-bf90-97dc8a54d503 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.033284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506a466-fbb1-4708-bf90-97dc8a54d503 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.035791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 137/270] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:25.037414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506a466-fbb1-4708-bf90-97dc8a54d503 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.063877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 134/271] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:25.065259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.065557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.065834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.098731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:25.099938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:25.184944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 135/272] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:25.186662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.191388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506a466-fbb1-4708-bf90-97dc8a54d503 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:25.246303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.247071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.247456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.247836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.248966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] RBAC: Adding query filter params (role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=f358d57309cb45c881901c818e27429b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:25.249253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:25.250808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:25.257655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 138/273] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:25 2022] GET /identity/v3/role_assignments?user.id=f358d57309cb45c881901c818e27429b&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:25.258606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.294793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:25.305089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:25.312116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a268-ae62-4e53-89ef-a4be555a51c3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:25.349937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.351046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.351541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.351991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.376716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:25.385134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:25.386095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:25.386861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:25.389439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 136/274] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:25.391723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd827e-0b8e-4d74-b05e-5e215c5f031e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:25.397810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.491696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.492554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.492837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.493108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.494582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:25.501886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:25.503758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 139/275] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:25.511145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.531258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d567c-eb17-47f9-90cd-3b584b70e7bf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:25.579471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.580225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.580425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.580720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.582572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:25.583065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:25.584269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:25.596557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 137/276] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:25.612422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.632240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:25.637564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e38747-d259-4cd1-994a-9245c8a076c9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:25.639227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:25.678781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df557c36-fca8-4259-8491-f582b68fa8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.679798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df557c36-fca8-4259-8491-f582b68fa8d9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.680350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df557c36-fca8-4259-8491-f582b68fa8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.680981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df557c36-fca8-4259-8491-f582b68fa8d9 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.682219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df557c36-fca8-4259-8491-f582b68fa8d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:25.684566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df557c36-fca8-4259-8491-f582b68fa8d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:25.741887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379044f-4bbe-4fa7-a19b-3c9d69b85f06 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:25.795314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:25.796200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.796485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.796785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] PATH_INFO: `/v3/users/5b0ce028be10405e834e4fa616dddf7b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.816349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] RBAC: Authorizing `identity:get_user(user_id=5b0ce028be10405e834e4fa616dddf7b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:25.818443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:25.820587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 138/277] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:25 2022] GET /identity/v3/users/5b0ce028be10405e834e4fa616dddf7b => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:25.824969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 140/278] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:25 2022] POST /identity/v3/users => generated 293 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:25.833596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:25.886778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfa12e-8da5-4873-959d-dafe9df1ed5b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.886778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfa12e-8da5-4873-959d-dafe9df1ed5b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.886778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfa12e-8da5-4873-959d-dafe9df1ed5b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.888639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 141/279] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:25.894839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:25.895967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:25.896473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:25.934157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:25.940089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:25.942307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:25.974187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19205b6f-9678-4bc7-a406-5fd995ab4f7e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.042695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.044830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.045130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.045468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.046264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, user.id=5b0ce028be10405e834e4fa616dddf7b, scope.system=all) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:26.046735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:26.048020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:26.056741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 142/280] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:26.064704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 139/281] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Jul 18 10:22:25 2022] GET /identity/v3/role_assignments?user.id=5b0ce028be10405e834e4fa616dddf7b&scope.system=all&role.id=19944ab23542462988b68290167ac622 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:26.083551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:26.195383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:26.214642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a700679-4ab8-4dbd-983b-d4ed59197195 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.257120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.257924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.258210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.258490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.272474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:26.283701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:26.284630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:26.285450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:26.289057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 143/282] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:26.294526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:26.405787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b43b4f-0cc5-4c5f-bed0-10a7dd7d7e8b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.460016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.461156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.461644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.462304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.463942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:26.464456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:26.466051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:26.476942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:26.480650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 140/283] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:26.486304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:26.607562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e43ca5-2a31-4a50-9cd2-fd5713fb12d7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.644764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.646001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.646490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.646991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] PATH_INFO: `/v3/users/2c34492ca5a649cb959a470ba3832475` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.648399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=2c34492ca5a649cb959a470ba3832475)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:26.650624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:26.652983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 144/284] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:26 2022] GET /identity/v3/users/2c34492ca5a649cb959a470ba3832475 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:26.658112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:26.772625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b453f-c526-43ed-bd88-083b1be9da5c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.821170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3607-0d13-469c-91a4-ded13441c770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.822642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3607-0d13-469c-91a4-ded13441c770 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.823134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3607-0d13-469c-91a4-ded13441c770 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.823636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3607-0d13-469c-91a4-ded13441c770 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.825091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b3607-0d13-469c-91a4-ded13441c770 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:26.826732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b3607-0d13-469c-91a4-ded13441c770 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:26.828721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 141/285] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:26 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:26.834693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:26.851827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:26.948022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd9d9-c94b-4a74-ac26-137d27b43ea2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.992236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7849f9-ef35-41ac-8dc3-7e2245c21c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.994382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7849f9-ef35-41ac-8dc3-7e2245c21c1d None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.994909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7849f9-ef35-41ac-8dc3-7e2245c21c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.995387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7849f9-ef35-41ac-8dc3-7e2245c21c1d None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/2c34492ca5a649cb959a470ba3832475/roles/a0cbbd58db464ac994fa37db8c9b9fdd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.997847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7849f9-ef35-41ac-8dc3-7e2245c21c1d None admin] RBAC: Authorizing `identity:create_grant(project_id=7d83853bf6704e0fa8f738c857c951c2, user_id=2c34492ca5a649cb959a470ba3832475, role_id=a0cbbd58db464ac994fa37db8c9b9fdd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:26.999530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7849f9-ef35-41ac-8dc3-7e2245c21c1d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:27.075933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 145/286] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:22:26 2022] PUT /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2/users/2c34492ca5a649cb959a470ba3832475/roles/a0cbbd58db464ac994fa37db8c9b9fdd => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:27.377270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f4d3-3cb1-4282-9822-9ab0c99b8bae None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:27.377655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f4d3-3cb1-4282-9822-9ab0c99b8bae None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:27.377833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f4d3-3cb1-4282-9822-9ab0c99b8bae None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:27.379229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 142/287] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:27.385435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:27.385919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:27.386380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:27.408652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:27.410074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:27.497327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 146/288] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:27.532266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f4d3-3cb1-4282-9822-9ab0c99b8bae None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:27.633051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:27.664744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f4d3-3cb1-4282-9822-9ab0c99b8bae None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:27.700120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:27.700944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:27.701230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:27.701506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:27.712426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:27.720078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:27.721039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:27.721837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 143/289] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:27.729057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:27.732996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:27.849111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6435f-9908-4725-9623-1491f08648f7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:27.890850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:27.891522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:27.891811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:27.892091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:27.892881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:27.893197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:27.894579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:27.912107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 147/290] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:27.918529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:28.034025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420a54-dda6-4a8e-b844-1a069eeb7fd1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:28.062654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:28.070597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:28.071553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.071943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.072231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] PATH_INFO: `/v3/users/e8d821ab686f49b3b9a3decc1d1cc389` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.073648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] RBAC: Authorizing `identity:get_user(user_id=e8d821ab686f49b3b9a3decc1d1cc389)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:28.075118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:28.077363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 144/291] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:27 2022] GET /identity/v3/users/e8d821ab686f49b3b9a3decc1d1cc389 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:28.083433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:28.148875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:28.207124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32da1e-d836-4aa5-81a3-9bfa01b06413 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:28.263753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82630144-a50e-4f40-95e6-1f4004ac4610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:28.263753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630144-a50e-4f40-95e6-1f4004ac4610 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.263753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630144-a50e-4f40-95e6-1f4004ac4610 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.263753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630144-a50e-4f40-95e6-1f4004ac4610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.334818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82630144-a50e-4f40-95e6-1f4004ac4610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:28.336356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82630144-a50e-4f40-95e6-1f4004ac4610 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:28.397596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:28.437711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 148/292] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:28.443211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:28.549284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82bd9c-5f4e-44fd-8aa0-f5cab3f6d53b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:28.569881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf734-1fb8-4468-a002-386c8b19d155 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.570487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf734-1fb8-4468-a002-386c8b19d155 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.570984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf734-1fb8-4468-a002-386c8b19d155 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.573648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 149/293] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:28.578768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cd80f-c627-46a3-8aff-079698586c0a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.579277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cd80f-c627-46a3-8aff-079698586c0a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.579754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cd80f-c627-46a3-8aff-079698586c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.584556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:28.585491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.585778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.586113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.588971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:28.590438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:28.592303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 145/294] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:28 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:28.598582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:28.603067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-088cd80f-c627-46a3-8aff-079698586c0a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:28.604468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-088cd80f-c627-46a3-8aff-079698586c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:28.688340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:28.699164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 150/295] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:28.700524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f07897-1077-4955-81a4-d95750705261 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.700844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f07897-1077-4955-81a4-d95750705261 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.701127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f07897-1077-4955-81a4-d95750705261 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.702670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 151/296] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:28.708288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.708582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.708897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.725615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:28.726816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:28.752267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3dff-5338-4a8c-9d76-385f23366e8d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:28.789021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:28.789683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.790038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.790888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.792012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=e8d821ab686f49b3b9a3decc1d1cc389) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:28.792364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:28.793965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57b2ef-16f3-4613-998c-19da048d5963 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:28.797723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 152/297] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:28.802351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:28.804949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 146/298] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:28 2022] GET /identity/v3/role_assignments?user.id=e8d821ab686f49b3b9a3decc1d1cc389&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=19944ab23542462988b68290167ac622 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:28.819080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f8a-5749-49cc-a33a-e5eb7856a56d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.819383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f8a-5749-49cc-a33a-e5eb7856a56d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.819653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f8a-5749-49cc-a33a-e5eb7856a56d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.821185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 147/299] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:28.834237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.834851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.835141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.846313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:28.862118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:28.863342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:28.948389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713eb39-b72d-4481-8cda-4efd0dc2c97e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:28.955956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 148/300] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:28.957173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:28.994068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:28.995128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.995414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.995686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.006968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:29.016890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:29.018666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 153/301] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:29.019438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.044665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.095536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094fade-86d6-4a44-a32a-aa0f1c2b25d1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.126796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.135170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.135969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.136256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.136535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.137752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.139175 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.140821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 149/302] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:29.141589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.144519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef1512-6ebd-4136-8c75-55bac1cffe6e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.184578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.185421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.185679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.185958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.197993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:29.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:29.206693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 154/303] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:29.213320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.262257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb82d9-9bda-405a-88a9-5779c0e85f78 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.288789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.299122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.299794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.300079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.300357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.301145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:29.301455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.302836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.311941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 150/304] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:29.312851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.338813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a8d92-9267-4fe0-8a66-3bc75593113e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.377381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.378051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.378336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.378654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] PATH_INFO: `/v3/groups` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.379316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.380781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.422842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba900897-766b-4f4c-9c6f-9b64699ddde8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.456123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.456751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.457043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.457583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.458389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:29.458705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.460080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.467561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 155/305] 10.222.0.47 () {60 vars in 1134 bytes} [Mon Jul 18 10:22:29 2022] POST /identity/v3/groups => generated 229 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:29.468730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.472731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 151/306] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:29.475202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.483218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.558212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.622538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986454a9-8dde-4052-931f-b96d2912216b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.633166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601ca73-ece1-4833-9b5a-ac72d8859fed None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.670229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.671199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.671482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.671761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] PATH_INFO: `/v3/users/5c796b64525c48268fa9ae19d7d78981` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.682144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.683167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.683611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.683895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] PATH_INFO: `/v3/users/5b0ce028be10405e834e4fa616dddf7b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.685680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] RBAC: Authorizing `identity:get_user(user_id=5b0ce028be10405e834e4fa616dddf7b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.687954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.688407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] RBAC: Authorizing `identity:get_user(user_id=5c796b64525c48268fa9ae19d7d78981)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.690542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 152/307] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/users/5b0ce028be10405e834e4fa616dddf7b => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:29.690975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.692120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 156/308] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/users/5c796b64525c48268fa9ae19d7d78981 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:29.697914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.699789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.778982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.812514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c1d0a-d739-4163-b1f5-55759304e17d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.825210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac564-2e7e-4f1a-ae4f-8dbc6ccdc598 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.831389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.850425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.851479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.851961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.852451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.853869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] RBAC: Adding query filter params (role.id=a0cbbd58db464ac994fa37db8c9b9fdd, user.id=5c796b64525c48268fa9ae19d7d78981, scope.system=all) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:29.854393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.856151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21249b3-0a33-4728-84a3-8fe0394ffbd8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.861828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.869027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.869027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.869027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] PATH_INFO: `/v3/system/users/5b0ce028be10405e834e4fa616dddf7b/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.869027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5b0ce028be10405e834e4fa616dddf7b, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.869027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.869659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 153/309] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/role_assignments?user.id=5c796b64525c48268fa9ae19d7d78981&scope.system=all&role.id=a0cbbd58db464ac994fa37db8c9b9fdd => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:29.892634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd5342-1e95-4bd3-8dd1-258f2f70547c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.893071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd5342-1e95-4bd3-8dd1-258f2f70547c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.893367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd5342-1e95-4bd3-8dd1-258f2f70547c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.894791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 154/310] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:29.900881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.901297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.901585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.921012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:29.922841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:29.961509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 157/311] 10.222.0.47 () {58 vars in 1334 bytes} [Mon Jul 18 10:22:29 2022] PUT /identity/v3/system/users/5b0ce028be10405e834e4fa616dddf7b/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:30.020284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 155/312] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:30.045155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:30.158811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab24d7d-e4e4-4646-8d3b-a1a5672e4a39 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:30.187300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:30.196804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:30.197748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:30.198043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:30.198324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:30.210027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:30.218257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:30.219172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:30.219936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 158/313] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:30.226046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:30.340574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07aa4c-6dc5-4f5c-853e-1dfc67213176 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:30.354694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:30.375901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:30.376583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:30.376886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:30.377165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:30.377918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:30.378227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:30.379583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:30.387827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 156/314] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:30.394060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:30.502390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52a95-d387-438f-b596-5af1035d45ff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:30.546986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:30.548248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:30.548771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:30.549432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] PATH_INFO: `/v3/users/2c34492ca5a649cb959a470ba3832475` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:30.550836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] RBAC: Authorizing `identity:get_user(user_id=2c34492ca5a649cb959a470ba3832475)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:30.552620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:30.554775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 159/315] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:30 2022] GET /identity/v3/users/2c34492ca5a649cb959a470ba3832475 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:30.561063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:30.669127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14b1c-7143-4209-961b-6692e0c75e04 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:30.706862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:30.708670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:30.709318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:30.709879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:30.755213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:30.782216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:30.784012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:30.885709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 157/316] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:30.893768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:30.950959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:30.996248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463c6a1-8629-42bc-82bb-62ab8e4d56fb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:31.033174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc666e0b-b8e3-43bd-aed2-c0047d3a16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:31.034107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666e0b-b8e3-43bd-aed2-c0047d3a16bd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.034386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666e0b-b8e3-43bd-aed2-c0047d3a16bd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.034659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666e0b-b8e3-43bd-aed2-c0047d3a16bd None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.035871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc666e0b-b8e3-43bd-aed2-c0047d3a16bd None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:31.037361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc666e0b-b8e3-43bd-aed2-c0047d3a16bd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:31.039090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 160/317] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:30 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:31.045871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:31.126466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:31.154873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d81b-65d7-4287-81af-81eecc6b2304 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:31.192424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:31.193136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.193423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.193700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.194536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] RBAC: Adding query filter params (role.id=a0cbbd58db464ac994fa37db8c9b9fdd, scope.project.id=7d83853bf6704e0fa8f738c857c951c2, user.id=2c34492ca5a649cb959a470ba3832475) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:31.194850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:31.196781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecbebf-4360-447d-a007-004913d3a2d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:31.204042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 158/318] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:31 2022] GET /identity/v3/role_assignments?user.id=2c34492ca5a649cb959a470ba3832475&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=a0cbbd58db464ac994fa37db8c9b9fdd => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:31.606632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238a24-5bcf-4348-8dea-6c4d62e5efc1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.607012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238a24-5bcf-4348-8dea-6c4d62e5efc1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.607209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238a24-5bcf-4348-8dea-6c4d62e5efc1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.608602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 161/319] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:31.613852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.614146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.614430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.631522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:31.632737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:31.702132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 159/320] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:31.724635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238a24-5bcf-4348-8dea-6c4d62e5efc1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:31.826552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238a24-5bcf-4348-8dea-6c4d62e5efc1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:31.849972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:31.861517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa052e-2e7f-49e4-9d8f-35cf863bd126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:31.862281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa052e-2e7f-49e4-9d8f-35cf863bd126 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.862573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa052e-2e7f-49e4-9d8f-35cf863bd126 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.862850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa052e-2e7f-49e4-9d8f-35cf863bd126 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.864071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa052e-2e7f-49e4-9d8f-35cf863bd126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:31.865543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa052e-2e7f-49e4-9d8f-35cf863bd126 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:31.867165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 162/321] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:31.873219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:31.943202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:31.976339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e66e68-b5f2-4e18-aef8-c216ea34f8a8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.004395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29864a-75a6-43eb-bbb0-86d6ff3816c5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.005060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29864a-75a6-43eb-bbb0-86d6ff3816c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.005269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29864a-75a6-43eb-bbb0-86d6ff3816c5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.006939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 163/322] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:32.011939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d50a-91c7-46d9-8bc5-15abb6c639ee None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.012314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d50a-91c7-46d9-8bc5-15abb6c639ee None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.012666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d50a-91c7-46d9-8bc5-15abb6c639ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.018223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.020025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.020469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.020921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] PATH_INFO: `/v3/groups` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.021792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.023379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.030767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4f30d50a-91c7-46d9-8bc5-15abb6c639ee None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:32.031966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f30d50a-91c7-46d9-8bc5-15abb6c639ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:32.098768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 164/323] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:32.100195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074fad0-8595-46bc-abd5-08b68a394b0f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.100501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074fad0-8595-46bc-abd5-08b68a394b0f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.100818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074fad0-8595-46bc-abd5-08b68a394b0f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.102062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 165/324] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:32.107623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.107938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.108213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.124051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:32.125218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:32.189001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 166/325] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:32.189795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.236860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:32.286576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868153b-8fcd-42c3-b39a-0cd69e30f74a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.319127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.319908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.320200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.320469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.331052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:32.339223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:32.340279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 167/326] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:32.341320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.377572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 160/327] 10.222.0.47 () {60 vars in 1134 bytes} [Mon Jul 18 10:22:31 2022] POST /identity/v3/groups => generated 226 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:32.378324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.461468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97512bbf-4cbc-4b91-8985-19245299eccd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.474623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:32.492548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1dd35-3ca1-4353-9e09-225cd1b786d2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.501325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.502140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.502428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.502708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.516098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:32.521726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:32.526571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:32.527455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:32.535394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 168/328] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:32.536210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.536309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.536395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.536395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.536707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.538434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:32.538434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.539901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.557052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 161/329] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:32.564197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.658953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1244a6-be9e-467e-b8e9-4a4e64bcb277 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.692464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540af2-4d3c-4f57-a4b2-ef7bbf354a58 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.711482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.715535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.715535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.715535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.715535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:32.715535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.717643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.726316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 169/330] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:32.731897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.739359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.739359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.739359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.739359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] PATH_INFO: `/v3/users/5c796b64525c48268fa9ae19d7d78981` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.740623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=5c796b64525c48268fa9ae19d7d78981)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.742160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.748436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 162/331] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/users/5c796b64525c48268fa9ae19d7d78981 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:32.749934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.780319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:32.826714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:32.872170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c081-d2e6-43bd-b51a-8312beabe097 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.878893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51078c-f3f3-4310-899f-76b561164ce8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.913925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.914856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.915189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.915575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] PATH_INFO: `/v3/users/5b0ce028be10405e834e4fa616dddf7b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.916836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] RBAC: Authorizing `identity:get_user(user_id=5b0ce028be10405e834e4fa616dddf7b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.918294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.920253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 170/332] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/users/5b0ce028be10405e834e4fa616dddf7b => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:32.920595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d2c0-097a-4eb6-ad58-c9435dc7d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.921561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d2c0-097a-4eb6-ad58-c9435dc7d1ff None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.921850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d2c0-097a-4eb6-ad58-c9435dc7d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.922130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d2c0-097a-4eb6-ad58-c9435dc7d1ff None admin] PATH_INFO: `/v3/system/users/5c796b64525c48268fa9ae19d7d78981/roles/a0cbbd58db464ac994fa37db8c9b9fdd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.923660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855d2c0-097a-4eb6-ad58-c9435dc7d1ff None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5c796b64525c48268fa9ae19d7d78981, role_id=a0cbbd58db464ac994fa37db8c9b9fdd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.926105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:32.926761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855d2c0-097a-4eb6-ad58-c9435dc7d1ff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.997645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 163/333] 10.222.0.47 () {58 vars in 1334 bytes} [Mon Jul 18 10:22:32 2022] PUT /identity/v3/system/users/5c796b64525c48268fa9ae19d7d78981/roles/a0cbbd58db464ac994fa37db8c9b9fdd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:33.044948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:33.048315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda2ac-9e22-4de0-b859-3edd84315434 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:33.094959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:33.095672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:33.096008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:33.096289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:33.097172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] RBAC: Adding query filter params (role.id=19944ab23542462988b68290167ac622, user.id=5b0ce028be10405e834e4fa616dddf7b, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:33.097534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:33.098993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd596c-cc06-4885-9c0d-c11d14c72ff8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:33.113038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 171/334] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Jul 18 10:22:32 2022] GET /identity/v3/role_assignments?user.id=5b0ce028be10405e834e4fa616dddf7b&scope.system=all&role.id=19944ab23542462988b68290167ac622 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:34.886650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772ebff-bc08-459c-8d5d-8dda74df414d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.887189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772ebff-bc08-459c-8d5d-8dda74df414d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.887189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772ebff-bc08-459c-8d5d-8dda74df414d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.888906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 164/335] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:34.894454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.894876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.895060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.913147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:34.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:34.939476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399aa3b-0acd-4f4f-ada9-449b39ada5fe None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.939793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399aa3b-0acd-4f4f-ada9-449b39ada5fe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.940048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399aa3b-0acd-4f4f-ada9-449b39ada5fe None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.941586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 165/336] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:34.947950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.948248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.948535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.978222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:34.979418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:35.004632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 172/337] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:35.066049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:35.111038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 166/338] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:35.179082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:35.201054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:35.215110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e874d8-c468-4cd6-8a61-e99b8bad1027 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:35.255297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:35.256110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:35.256403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:35.256702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:35.275517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:35.293766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:35.300270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:35.301107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 173/339] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:35.312056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:35.325824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:35.387745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3500836-6819-4bc8-884e-130160e1d841 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:35.489493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:35.490250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:35.490535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:35.490812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:35.515731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:35.523960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:35.525741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:35.526723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:35.536312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 167/340] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:35.545307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:35.566231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dae45-b587-4910-a4a1-c5aca41592b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:35.653408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:35.654390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:35.654825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:35.655256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:35.656179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:35.656642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:35.658221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:35.674925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 174/341] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:35.684409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:35.697306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f21d5-5f04-4828-a582-1189fa16ad98 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:35.709479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:35.715140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:35.746613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:35.747560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:35.748062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:35.748563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:35.749518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:35.750112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:35.751640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:35.768243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 168/342] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:35.773576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:35.853524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade583-4487-4872-8ef1-effd4b5508b7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:35.929844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:35.931606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:35.932045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:35.932585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:35.935433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed100c-1636-4bc2-bcf7-775ad29f08ce None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:35.976690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86066c-dd4a-4eef-ad34-a45d78549c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:35.977356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86066c-dd4a-4eef-ad34-a45d78549c4d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:35.977680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86066c-dd4a-4eef-ad34-a45d78549c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:35.977958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86066c-dd4a-4eef-ad34-a45d78549c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.023931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.062733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.064271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.065953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86066c-dd4a-4eef-ad34-a45d78549c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.067420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86066c-dd4a-4eef-ad34-a45d78549c4d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.088517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.186766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 175/343] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:22:36.192348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.230921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 169/344] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:36.232108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fd0ed-3eaa-4a06-94cb-f89f46e2ca43 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.232406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fd0ed-3eaa-4a06-94cb-f89f46e2ca43 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.232710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fd0ed-3eaa-4a06-94cb-f89f46e2ca43 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.234240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 170/345] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:36.239159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.239456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.239742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.271579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:36.271579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:36.355204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438025d-6a5d-400a-a53e-522db3b69748 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:36.373147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.380799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 171/346] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:36.381980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.402903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.409767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed355426-46db-4822-8b64-e28b6b9c9964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:36.411019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355426-46db-4822-8b64-e28b6b9c9964 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.411019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355426-46db-4822-8b64-e28b6b9c9964 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.411205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355426-46db-4822-8b64-e28b6b9c9964 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.413177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed355426-46db-4822-8b64-e28b6b9c9964 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.416865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed355426-46db-4822-8b64-e28b6b9c9964 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.418968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 176/347] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:36.420350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8429900-4720-4dd6-8c2e-7e1f0e69c374 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.422392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8429900-4720-4dd6-8c2e-7e1f0e69c374 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.422868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8429900-4720-4dd6-8c2e-7e1f0e69c374 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.427873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 177/348] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:36.427873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8429900-4720-4dd6-8c2e-7e1f0e69c374 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.534369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950248ca-6aa5-4a70-a04a-0f867ddc323f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:36.552936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8429900-4720-4dd6-8c2e-7e1f0e69c374 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:36.581763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:36.582901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.583627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.583914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.585431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.588418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.591086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 172/349] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:36.592564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.601404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b403f-d1fc-41a9-8465-e72cca52650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:36.602690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b403f-d1fc-41a9-8465-e72cca52650f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.603427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b403f-d1fc-41a9-8465-e72cca52650f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.604121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b403f-d1fc-41a9-8465-e72cca52650f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.618203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b403f-d1fc-41a9-8465-e72cca52650f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bc4b403f-d1fc-41a9-8465-e72cca52650f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:36.627169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:36.628048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:22:36.629446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:36.631853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 178/350] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:36.633346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.634414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.634896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.655148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:36.657460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:36.682130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.692803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.725308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b486-d4f7-4fac-a081-15842f957beb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:36.739033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 179/351] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:36.740217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.783866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:36.786825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.787171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.787472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.796203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.799101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.799798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 173/352] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:36.800579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.880362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e7c9f-ca14-4795-b675-638f77883a50 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:36.928030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6040cdf-6ff9-46a4-84be-b738634e39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:36.929083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040cdf-6ff9-46a4-84be-b738634e39dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.929467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040cdf-6ff9-46a4-84be-b738634e39dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.929846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040cdf-6ff9-46a4-84be-b738634e39dd None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.931682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6040cdf-6ff9-46a4-84be-b738634e39dd None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.932229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7c3fc-a5c5-4359-8ba9-0de6bdbd1609 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:36.933229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6040cdf-6ff9-46a4-84be-b738634e39dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.935037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 180/353] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:36.936216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a7b86-92f8-4d03-bace-05b45b0665de None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.936390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a7b86-92f8-4d03-bace-05b45b0665de None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.936717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a7b86-92f8-4d03-bace-05b45b0665de None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.939114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 181/354] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:36.939838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a7b86-92f8-4d03-bace-05b45b0665de None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.973941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:36.974621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:36.974912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:36.975193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:36.976904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:36.978048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:36.979907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:36.982586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:36.994498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 174/355] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:36.994894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:36.997417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.080667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a7b86-92f8-4d03-bace-05b45b0665de None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.125387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.126209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.126495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.126778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.139786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:37.139940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbad77-76e5-4cd8-a7e9-e586031d98aa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:37.147639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:37.148710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 182/356] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:37.149855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.183108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.183770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.184054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.184333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.185208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=19944ab23542462988b68290167ac622, scope.project.id=61a5cd521b194aab81a0ff7939ebd317) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:37.185526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:37.186974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25c8ce-404e-482d-8ba0-3e44ead37876 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:37.192723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 175/357] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:36 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=19944ab23542462988b68290167ac622 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:37.198512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.198832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.199081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.220109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:37.222417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:37.271660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.281999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6f4c-cc3d-48fe-97d0-d4a93f58b221 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.305555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.322250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 176/358] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:37.324221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.332993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.332993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.333922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.334316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.335296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=61a5cd521b194aab81a0ff7939ebd317) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:37.335721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:37.338129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feea009-ce4e-4082-a2f7-15e3f20e4c07 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:37.346458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 183/359] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:37.348192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5734ccc-5a17-40aa-88b9-c31213933d96 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.348629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5734ccc-5a17-40aa-88b9-c31213933d96 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.349069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5734ccc-5a17-40aa-88b9-c31213933d96 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.355784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 184/360] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:37.357128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5734ccc-5a17-40aa-88b9-c31213933d96 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.456740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06a75-1151-4086-a1dd-1c219787ce4c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.495935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5734ccc-5a17-40aa-88b9-c31213933d96 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.507448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.508994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.509285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.509583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] PATH_INFO: `/v3/users/5c796b64525c48268fa9ae19d7d78981` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.512016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] RBAC: Authorizing `identity:get_user(user_id=5c796b64525c48268fa9ae19d7d78981)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:37.513793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:37.516633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 177/361] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/users/5c796b64525c48268fa9ae19d7d78981 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:37.517476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.550818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.551498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.551784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.552063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.552865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:37.553182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:37.554568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0844e-172f-4b6f-a7a6-ccd6c922dafc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:37.566759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 185/362] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:37.568342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.570263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.570563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.589650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:37.590861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:37.600918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.615377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.651048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26604e10-b881-41f1-b8d3-4eb95ad710de None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.671851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 186/363] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:37.678551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.707203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.708373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.708664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.709274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.724077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:37.734574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:37.735512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:37.736311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 178/364] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:37.737016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.810702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4384d93-cc39-4367-a235-a75f1d434e7b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.860575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.861276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.861826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.862112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.863464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] RBAC: Adding query filter params (role.id=a0cbbd58db464ac994fa37db8c9b9fdd, user.id=5c796b64525c48268fa9ae19d7d78981, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:37.864082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:37.865799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:37.883742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 187/365] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/role_assignments?user.id=5c796b64525c48268fa9ae19d7d78981&scope.system=all&role.id=a0cbbd58db464ac994fa37db8c9b9fdd => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:37.884520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf748d18-2c4a-4f52-9d32-2f2c5540dddf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.884727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.903526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.909170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.930391 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.932265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.933900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.934159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.018430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:38.020839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:38.126831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a243a1e-b1d8-470c-9eba-9089ab21295c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.164721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 179/366] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:38.166487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.185265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.186937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.187461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.187962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.202143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:38.211150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:38.212118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 188/367] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:38.213424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.217098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:38.325735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:38.345144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2df1-3072-4fad-956a-3a3a4e8808d8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.366441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e6db6-5c7e-4280-90f2-c9921d330b06 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.392037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.393083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.393598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.394171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.395125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:38.395640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:38.397474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:38.409844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 180/368] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:38.410529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.411404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.411512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.411689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.411973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.413456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:38.415295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:38.417818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 189/369] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:38.421658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.524969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:38.546801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766edc-a0ae-423f-aae4-391bfa562628 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.549819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a873b-62c9-44fe-947d-598d7a596fb1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.594183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.594409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.594618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.594831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.595706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:38.596021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:38.597736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:38.599384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.600753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.601264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.601753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.608309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 181/370] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:38.609262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.616882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:38.700571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:38.702428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:38.733375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5d81f-b92d-4dc2-a7c2-076b98bebcec None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.774709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.775666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.775892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.776170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.777492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:38.778931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:38.781154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 182/371] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:38.781994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.800100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:38.841357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 190/372] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:38.842668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:38.901843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:38.918866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e09354-6e42-4381-a0cb-f9145a9a77df None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:38.961818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:38.962913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:38.963349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:38.963800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:38.982145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01bd14-04e1-440d-91ea-ca01538c3ee6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.029018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.029612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.029899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.030178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.031028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=19944ab23542462988b68290167ac622, scope.project.id=7d83853bf6704e0fa8f738c857c951c2) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:39.031348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.033394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.042247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 191/373] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=19944ab23542462988b68290167ac622 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:39.043073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.056348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.058106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.077555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.170603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda1880-99e4-494c-9043-eb8096f14e05 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.174186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.196737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 183/374] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:39.204456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.213550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.214432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.214772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.215060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.216352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.218067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.220260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 192/375] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:39.225775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.346863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233a1fe-4af3-4380-b073-628735d15f4b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.384711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee9d9-9889-4ee0-926a-c7b83a114674 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.385012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.385853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.386192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.386483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.387701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.389248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.391266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 193/376] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:39.398661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.404998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.451369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.452239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.452524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.452831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] PATH_INFO: `/v3/projects/1a98445bbc6b4a039db83aaf6d202161` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.454699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] RBAC: Authorizing `identity:get_project(project_id=1a98445bbc6b4a039db83aaf6d202161)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.456150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.458323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 184/377] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/projects/1a98445bbc6b4a039db83aaf6d202161 => generated 344 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:39.465432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.483941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.568604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2de1b-1ffb-4277-bf8c-946e07f81d54 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.610129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.610725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.611014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.611301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.612258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=7d83853bf6704e0fa8f738c857c951c2) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:39.612586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.614238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035f672-f1fa-40b9-b817-46344653b376 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.622219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 194/378] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:39.623754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ebc0-05c4-41a7-a608-8a3e2229dc71 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.624060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ebc0-05c4-41a7-a608-8a3e2229dc71 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.624352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ebc0-05c4-41a7-a608-8a3e2229dc71 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.626025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 195/379] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:39.643494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.643807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.644074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.668417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec10b2-7b8e-48e2-bb48-bf3afa9117a0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.673016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:39.675351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:39.741653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.742609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.743425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.743725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] PATH_INFO: `/v3/groups/7aa7e009c70e458080cfe67b4b62b7d6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.758494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] RBAC: Authorizing `identity:get_group(group_id=7aa7e009c70e458080cfe67b4b62b7d6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.758494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.764347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 185/380] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/groups/7aa7e009c70e458080cfe67b4b62b7d6 => generated 229 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:39.770145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.808586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 196/381] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:39.837575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.851151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.899121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.927714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651278-4d69-420a-a951-72abd83fa826 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.972663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:39.974253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.974543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.974816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.975671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] RBAC: Adding query filter params (group.id=7aa7e009c70e458080cfe67b4b62b7d6, role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=1a98445bbc6b4a039db83aaf6d202161) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:39.975995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:39.977628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142092e-74fa-4492-be76-ac12b777bb6d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:39.981432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51769d0-febd-48a0-9c6b-2fea2781874b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.983949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 186/382] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/role_assignments?group.id=7aa7e009c70e458080cfe67b4b62b7d6&scope.project.id=1a98445bbc6b4a039db83aaf6d202161&role.id=e4cb7641166847b497698b97084ca96b => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:39.985337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fdb90-ba6a-4812-b34c-c89731b902e5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.985630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fdb90-ba6a-4812-b34c-c89731b902e5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.985911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fdb90-ba6a-4812-b34c-c89731b902e5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.987430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 187/383] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:39.998273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.998578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.998828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.028235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:40.030063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:40.031045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.031862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.032147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.032425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.058295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:40.067643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:40.068582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:40.069387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 197/384] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:40.080173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.170117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 188/385] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:40.192967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.198189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:40.207917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee039be-4003-4147-8b62-ec6bc94c6f80 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.241053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:40.253728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.254413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.254702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.254983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.255755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:40.256071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:40.257499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:40.274770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 198/386] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:40.285014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.371166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ec8b0-e274-426c-aeea-21ae07bc78fa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.435516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fc12e-bc3c-4ec2-95c4-0a9966445ed4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.437544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.438642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.439148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.439633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.460088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:40.469763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:40.470671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:40.471445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:40.476704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.476704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.476704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.476704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.478254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:40.478909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 189/387] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:40.479915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:40.481951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 199/388] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:40.484285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.487279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.541967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:40.679101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d615e-eef8-49f6-8844-d05e8ed66476 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.681500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:40.710098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c13792-f8a8-404a-a4f6-426c77832d77 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.734606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.735293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.735580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.735859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.736621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:40.737004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:40.738385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:40.753647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 190/389] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:40.759232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.764874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.765746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.766034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.766313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.769458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:40.771132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:40.773543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 200/390] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:40.781084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:40.900975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771238a0-a5fd-4bd7-8e84-92d090ab796f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.953628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:40.959553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.960483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.960750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.961042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.964648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:40.966173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:40.968185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 191/391] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:40.968755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ad6b6-b680-4321-9e83-f168acbf0ecb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:40.981228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:41.012725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:41.013626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:41.013913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:41.014191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317/groups/48721632e6eb4c6991278027d7f8e8bf/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:41.022046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] RBAC: Authorizing `identity:create_grant(project_id=61a5cd521b194aab81a0ff7939ebd317, group_id=48721632e6eb4c6991278027d7f8e8bf, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:41.023575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:41.061095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:41.110309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481c676-e2b3-4bb2-b4f4-7558a387a20a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:41.120773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 201/392] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Jul 18 10:22:40 2022] PUT /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317/groups/48721632e6eb4c6991278027d7f8e8bf/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:22:41.164004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9387d98-b536-48a1-930d-a18d22d1a9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:41.165627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9387d98-b536-48a1-930d-a18d22d1a9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:41.165898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9387d98-b536-48a1-930d-a18d22d1a9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:41.166149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9387d98-b536-48a1-930d-a18d22d1a9f2 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:41.173758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9387d98-b536-48a1-930d-a18d22d1a9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:41.175248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9387d98-b536-48a1-930d-a18d22d1a9f2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:41.177548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 192/393] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:40 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:41.183857 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:41.323773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e55d6b-1b23-468b-bd2e-01f32790d613 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:41.365650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cc264-1f67-4596-96a5-0acd052f53ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:41.366546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc264-1f67-4596-96a5-0acd052f53ae None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:41.366831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc264-1f67-4596-96a5-0acd052f53ae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:41.367109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc264-1f67-4596-96a5-0acd052f53ae None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317/groups/48721632e6eb4c6991278027d7f8e8bf/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:41.369535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cc264-1f67-4596-96a5-0acd052f53ae None admin] RBAC: Authorizing `identity:create_grant(project_id=61a5cd521b194aab81a0ff7939ebd317, group_id=48721632e6eb4c6991278027d7f8e8bf, role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:41.371061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cc264-1f67-4596-96a5-0acd052f53ae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:41.466783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 202/394] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Jul 18 10:22:41 2022] PUT /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317/groups/48721632e6eb4c6991278027d7f8e8bf/roles/6413d71218d344e8b26c8577cac9c4dc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:41.591487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c0927-aa10-4570-b48a-a08a11be3a6d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:41.591782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c0927-aa10-4570-b48a-a08a11be3a6d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:41.591977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c0927-aa10-4570-b48a-a08a11be3a6d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:41.593412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 193/395] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:41.600080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:41.600383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:41.600669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:41.622801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:41.624041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:41.629991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:41.717450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 203/396] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:41.751296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c0927-aa10-4570-b48a-a08a11be3a6d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:41.877508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:41.884465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c0927-aa10-4570-b48a-a08a11be3a6d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:41.928190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:41.929026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:41.929314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:41.929589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:41.950663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:41.958804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:41.959670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:41.960452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:41.965057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 194/397] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:41.967055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.155127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6abdf-d916-4850-a494-7bec7dcba2c6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.195245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.195955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.196840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.197093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.197963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:42.198278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:42.199651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57567-4b62-4b35-a9a2-83d6cea9f3ab None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:42.213342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 204/398] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:42.214372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.388572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef5ff5-fac8-47a8-947e-674cdea13234 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.407761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dbc3-ea0c-486b-ba9d-3baeb408cb80 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.408238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dbc3-ea0c-486b-ba9d-3baeb408cb80 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.408238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dbc3-ea0c-486b-ba9d-3baeb408cb80 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.410139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 205/399] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:42.418257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd0d42-71f2-4622-bfb3-08cfdcd74396 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.418557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd0d42-71f2-4622-bfb3-08cfdcd74396 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.418841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd0d42-71f2-4622-bfb3-08cfdcd74396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.437244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.438114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.438242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:42.438413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.438667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.439877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:42.441393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:42.443328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 195/400] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:42 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:42.452490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d4dd0d42-71f2-4622-bfb3-08cfdcd74396 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:42.452664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.452768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d4dd0d42-71f2-4622-bfb3-08cfdcd74396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:42.476355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:42.564894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 206/401] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:42.566230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21590e-54aa-4d11-bcd2-220b514fdcbf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.566518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21590e-54aa-4d11-bcd2-220b514fdcbf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.566795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21590e-54aa-4d11-bcd2-220b514fdcbf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.567503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10fc64-0ba5-480a-a5b1-b6996a2cc6e4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.568207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 207/402] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:42.573330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.573617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.573896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.590885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:42.592064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:42.607929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.609104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.609397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.610022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.611573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:42.613116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:42.616097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 196/403] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:42 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:42.617454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.668841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 208/404] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:42.669756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.745138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f274-db11-4b57-9e8b-d46c790b032d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.747606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:42.748076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:42.781194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.782003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.782289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.782573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.788747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689c6f0-0bed-45a3-a771-0225cf6fc421 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.794142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:42.802475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:42.803543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:42.804412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 197/405] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:42.805142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.827911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.829077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.829511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.829786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2/groups/48721632e6eb4c6991278027d7f8e8bf/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.831739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d83853bf6704e0fa8f738c857c951c2, group_id=48721632e6eb4c6991278027d7f8e8bf, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:42.833461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:42.916436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87a347-4d3d-46d1-92a5-4e8168e779a0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.952388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 209/406] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Jul 18 10:22:42 2022] PUT /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2/groups/48721632e6eb4c6991278027d7f8e8bf/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:22:42.953496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.963409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.964601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.965067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.965547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.980974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:42.991499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:42.992476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:42.994270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:42.997476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 198/407] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:43.010060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.044866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:43.178231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d6cd-65ad-4c40-87e7-726c2b4e7f38 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.184499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860363a2-a03c-4405-8161-85b6c596088f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.209503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:43.225632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.226899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.227681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.228109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.229191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:43.229652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.231193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.231699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.232491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.232902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.233276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.235229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:43.235229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.236159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.246456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 210/408] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:43.252384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 199/409] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:43.252521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.259122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.344319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:43.378997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403526b5-ce96-4423-a19f-4b4c0a3c8036 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.383093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e649f-37de-4f09-bfd6-9d695ad8c29e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.424706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.424706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.424706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.424706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.425492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.425492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.425762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 211/410] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:43 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:43.432828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.441409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.442512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.442662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.442839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] PATH_INFO: `/v3/groups/7aa7e009c70e458080cfe67b4b62b7d6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.449431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] RBAC: Authorizing `identity:get_group(group_id=7aa7e009c70e458080cfe67b4b62b7d6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.456701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.456701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 200/411] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:43 2022] GET /identity/v3/groups/7aa7e009c70e458080cfe67b4b62b7d6 => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:43.458458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.541951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:43.575316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b177e-af38-45c7-b892-1149ac77bf42 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.642535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.644302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.645040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.645705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] PATH_INFO: `/v3/projects/1a98445bbc6b4a039db83aaf6d202161` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.650333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] RBAC: Authorizing `identity:get_project(project_id=1a98445bbc6b4a039db83aaf6d202161)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.655471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.666008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 201/412] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:43 2022] GET /identity/v3/projects/1a98445bbc6b4a039db83aaf6d202161 => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:43.667750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.685126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c7a51-12b0-419e-ac43-1c5b149ada72 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.717082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:43.751035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.753328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.754427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.755225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.757792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.761410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.766055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 212/413] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:43 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:43.775729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:43.928402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2629-d0c5-472e-abf4-38219ad95f83 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.959156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fd015-215a-44b3-96eb-75a0bdcd6159 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:43.966313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8176d-f541-4f7e-89f4-762fbf4aa4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.967426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8176d-f541-4f7e-89f4-762fbf4aa4f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:43.967715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8176d-f541-4f7e-89f4-762fbf4aa4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:43.967998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8176d-f541-4f7e-89f4-762fbf4aa4f0 None admin] PATH_INFO: `/v3/projects/1a98445bbc6b4a039db83aaf6d202161/groups/7aa7e009c70e458080cfe67b4b62b7d6/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:43.970155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8176d-f541-4f7e-89f4-762fbf4aa4f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a98445bbc6b4a039db83aaf6d202161, group_id=7aa7e009c70e458080cfe67b4b62b7d6, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:43.971524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8176d-f541-4f7e-89f4-762fbf4aa4f0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:43.985005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:43.999031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872520bb-9ea9-44c8-a849-d556c528f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:43.999916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872520bb-9ea9-44c8-a849-d556c528f83a None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.000195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872520bb-9ea9-44c8-a849-d556c528f83a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.000468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872520bb-9ea9-44c8-a849-d556c528f83a None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2/groups/48721632e6eb4c6991278027d7f8e8bf/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.002685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872520bb-9ea9-44c8-a849-d556c528f83a None admin] RBAC: Authorizing `identity:create_grant(project_id=7d83853bf6704e0fa8f738c857c951c2, group_id=48721632e6eb4c6991278027d7f8e8bf, role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:44.004196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872520bb-9ea9-44c8-a849-d556c528f83a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:44.046378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 202/414] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Jul 18 10:22:43 2022] PUT /identity/v3/projects/1a98445bbc6b4a039db83aaf6d202161/groups/7aa7e009c70e458080cfe67b4b62b7d6/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:22:44.047530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0664f94-941c-4f85-a25d-7f71830d3270 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.047816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0664f94-941c-4f85-a25d-7f71830d3270 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.048094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0664f94-941c-4f85-a25d-7f71830d3270 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.049541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 203/415] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:44.050624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cd31-3592-4df5-af74-b37ab99dd80c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.050910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cd31-3592-4df5-af74-b37ab99dd80c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.051198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cd31-3592-4df5-af74-b37ab99dd80c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.052628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 204/416] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:44.055058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.055405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.055802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.075524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:44.077605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:44.089673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 213/417] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Jul 18 10:22:43 2022] PUT /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2/groups/48721632e6eb4c6991278027d7f8e8bf/roles/6413d71218d344e8b26c8577cac9c4dc => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:22:44.091620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.091916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.092193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.134267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:44.138819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:44.221229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:44.231960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 205/418] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:22:44.241868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 214/419] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:44.271700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.272204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.423388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952b621-b211-4943-9eaf-e5265830ecf6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:44.431568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa10ad-224c-4417-aa78-3be86caea217 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:44.451614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:44.485149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:44.485878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.486161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.486438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.497555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:44.505461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:44.505821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:44.506645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:44.507410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 206/420] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:44.509107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.509393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.509673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.516728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.543968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:44.553344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:44.554475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:44.558074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:44.565354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 215/421] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:44.570098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.665555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:44.677764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5db5-3e72-4a46-8c99-3a832218c00d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:44.714973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:44.715880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.716333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.716795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.718131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:44.718738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:44.720243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:44.731742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 207/422] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:44.738478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.811598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de05ae0-a0cf-472f-8719-ba11e62d83aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:44.887934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:44.889935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.891031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.891451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.893128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:44.894096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:44.896335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:44.897232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344efe2-f22b-4e1f-9fef-d211971c6888 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:44.916080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 216/423] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:44.922581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.945089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:44.967932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:44.968585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.968946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.969224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.986042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.072408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.072822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21e868-1ac3-499e-88d1-c42903270b93 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.074306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.108412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.109090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.109379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.109670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.178828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.180841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.221202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.227801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 208/424] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:45.240943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.297057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.300030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 217/425] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:45.306128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.416848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7045c-ce89-4bc5-84f9-d90c7b2cb979 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.449834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.457555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.458440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.458719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.458994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.460376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.462104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.464315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 218/426] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:45.475184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.534507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4926b2-95e4-47e8-bf32-371c1ece602e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.602586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.603739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.604026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.604304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] PATH_INFO: `/v3/projects/61a5cd521b194aab81a0ff7939ebd317` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.607958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] RBAC: Authorizing `identity:get_project(project_id=61a5cd521b194aab81a0ff7939ebd317)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.609431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.613615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 209/427] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/projects/61a5cd521b194aab81a0ff7939ebd317 => generated 298 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:45.624237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.667548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76d606-de94-468a-b71a-2801642f946f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.703108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81790e-7736-40bd-a1b7-d73b22094f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.703970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81790e-7736-40bd-a1b7-d73b22094f32 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.704259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81790e-7736-40bd-a1b7-d73b22094f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.704543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81790e-7736-40bd-a1b7-d73b22094f32 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.705760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81790e-7736-40bd-a1b7-d73b22094f32 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.707582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81790e-7736-40bd-a1b7-d73b22094f32 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.709583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 219/428] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:45.712271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8d18-09cc-4a36-931f-57cbeb3987e5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.712779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8d18-09cc-4a36-931f-57cbeb3987e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.713122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8d18-09cc-4a36-931f-57cbeb3987e5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.718617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 220/429] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:45.719438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f8d18-09cc-4a36-931f-57cbeb3987e5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.746898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.759941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395f543-5f30-41c5-a458-3a85c9f56667 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.792465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.802766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e09403-337c-4acf-9d3c-8fd1d92a1bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.803633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09403-337c-4acf-9d3c-8fd1d92a1bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.803919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09403-337c-4acf-9d3c-8fd1d92a1bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.804554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e09403-337c-4acf-9d3c-8fd1d92a1bd8 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.806951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09403-337c-4acf-9d3c-8fd1d92a1bd8 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.808719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e09403-337c-4acf-9d3c-8fd1d92a1bd8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.811110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 210/430] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:45.812501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.813024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.813308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.831593 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f8d18-09cc-4a36-931f-57cbeb3987e5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.835206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:45.836433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:45.868738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.869507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.869855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.870240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.871140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=61a5cd521b194aab81a0ff7939ebd317) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:45.871557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.873184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.880014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 221/431] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:45.880969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.924387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 211/432] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:45.947282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:45.997585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09e2d4-48d2-43f7-9b5e-327eb22825ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.030882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:46.036778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.037463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.037748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.038026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.038872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=19944ab23542462988b68290167ac622, scope.project.id=61a5cd521b194aab81a0ff7939ebd317) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:46.039186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:46.040980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:46.044247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:46.047871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 222/433] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=61a5cd521b194aab81a0ff7939ebd317&role.id=19944ab23542462988b68290167ac622 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:46.057778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91174737-ff76-42d7-b687-5cf6e662e30a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.097796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.098619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.098904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.099184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.111624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:46.119939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:46.123576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:46.125627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 212/434] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:46.132729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.267612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1509db-f512-4def-b300-00c26ea09fcf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.303874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.304542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.304852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.305164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.305987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:46.306310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:46.307734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b027136-4d0f-41cc-855b-fab8c1c3b92d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:46.317157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 223/435] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:46.323170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.354507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7516-5170-434c-9bbc-e81ae00dbe7b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.355072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7516-5170-434c-9bbc-e81ae00dbe7b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.355492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7516-5170-434c-9bbc-e81ae00dbe7b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.357444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 224/436] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:46.364775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.365299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.365751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.385810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:46.387020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:46.455063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:46.455669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba298b0f-c3f6-4b41-857a-b749ee1078fe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.482089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 225/437] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:46.498407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6668b4-f21d-49ed-b044-30c87899aaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.499058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6668b4-f21d-49ed-b044-30c87899aaeb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.499338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6668b4-f21d-49ed-b044-30c87899aaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.499613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6668b4-f21d-49ed-b044-30c87899aaeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.507711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.521325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:46.584849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6668b4-f21d-49ed-b044-30c87899aaeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:46.586115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6668b4-f21d-49ed-b044-30c87899aaeb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:46.640448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d9a71-6c8d-4972-a585-cef49687c036 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.679381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b1611-e55d-4d8c-9243-456632c627c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.680555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b1611-e55d-4d8c-9243-456632c627c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.681029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b1611-e55d-4d8c-9243-456632c627c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.681518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b1611-e55d-4d8c-9243-456632c627c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.693575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b1611-e55d-4d8c-9243-456632c627c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-242b1611-e55d-4d8c-9243-456632c627c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:46.701558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:46.702484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:46.703264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 226/438] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:46.705141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c70273-11e1-4879-a51c-f822a7348169 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.705141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c70273-11e1-4879-a51c-f822a7348169 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.705141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c70273-11e1-4879-a51c-f822a7348169 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.706367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 227/439] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:46.712191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c70273-11e1-4879-a51c-f822a7348169 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.719272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 213/440] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:46.722161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.722453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.722736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.741219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:46.742429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:46.772605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:46.794858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:46.817758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 214/441] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:46.818583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.827306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c70273-11e1-4879-a51c-f822a7348169 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.862899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.863574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.863857 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.864140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.864921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:46.865240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:46.866630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:46.874836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 228/442] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:46.877610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.919438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a95e1-0004-4781-8ecd-1eeb8b01b44e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.954306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:46.955169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:46.955448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:46.955725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:46.956931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:46.958627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:46.960304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 215/443] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:46.961091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:46.976329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a047297-5cb1-40c6-b33e-20589175cc79 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:46.987027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.006915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.010068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.010880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.011167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.011448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.021712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:47.030898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:47.031828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:47.032638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 229/444] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:47.033322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.058297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2afac-39ba-4f2c-9fdc-3247fb60f5aa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.093261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.093900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.094189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.094473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.131451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116c218-8a2c-42ed-8b12-06d16b5f1ba2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.161104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.162583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.164710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.165665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.165955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.166229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.167398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.168908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.170501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 230/445] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:47.173517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.207738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.215987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.267512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 216/446] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:47.268367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.277732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d59b-6b91-4659-b4e3-366774d5901a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.312178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.312799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.313086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.313362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.314111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:47.314422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.315793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.324322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 231/447] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:47.325143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.374372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825de9bb-cbcc-44b4-8fbc-845434066e17 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.407751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.408396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.408687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.408989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.409818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=19944ab23542462988b68290167ac622, scope.project.id=7d83853bf6704e0fa8f738c857c951c2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:47.410135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.411557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.417614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 217/448] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=19944ab23542462988b68290167ac622 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:47.418221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.431925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.434247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.437257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546c41c-62cf-43e7-a0bf-0e4135617658 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.474809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.476075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.476275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.476558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] PATH_INFO: `/v3/projects/1a98445bbc6b4a039db83aaf6d202161` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.477931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] RBAC: Authorizing `identity:get_project(project_id=1a98445bbc6b4a039db83aaf6d202161)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.479415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.481213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 232/449] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/projects/1a98445bbc6b4a039db83aaf6d202161 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:47.486998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.539957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd551ea-bd4f-4eee-8af1-37165ae2f46e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.584049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.584505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.584831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.585113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.612244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c14d6-f38c-42ec-b418-152b70e4c6fb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.660614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.661511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.661804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.662088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] PATH_INFO: `/v3/groups/7aa7e009c70e458080cfe67b4b62b7d6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.663276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] RBAC: Authorizing `identity:get_group(group_id=7aa7e009c70e458080cfe67b4b62b7d6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.664846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.666517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 233/450] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/groups/7aa7e009c70e458080cfe67b4b62b7d6 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:47.672425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.672562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.674142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.686202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.711271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.787861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2c81d-527b-43af-b3c4-0018014d88d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.794434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 218/451] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:47.800243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.825967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.826628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.826914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.827198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.828033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] RBAC: Adding query filter params (group.id=7aa7e009c70e458080cfe67b4b62b7d6, role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=1a98445bbc6b4a039db83aaf6d202161) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:47.828354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.829813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.835878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 234/452] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/role_assignments?group.id=7aa7e009c70e458080cfe67b4b62b7d6&scope.project.id=1a98445bbc6b4a039db83aaf6d202161&role.id=e4cb7641166847b497698b97084ca96b => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:47.903703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98f7e4-caff-4d5c-988d-057705e2f291 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.915986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.939992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.940904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.941189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.941465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] PATH_INFO: `/v3/projects/7d83853bf6704e0fa8f738c857c951c2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.942694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] RBAC: Authorizing `identity:get_project(project_id=7d83853bf6704e0fa8f738c857c951c2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.944141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.946441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 219/453] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/projects/7d83853bf6704e0fa8f738c857c951c2 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:47.952242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:48.052578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:48.055939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24e54-ff01-402b-a8b4-8d9b8a0de7f2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:48.088836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76abb77-7ce5-4d5c-b82b-3fe28ebdb562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:48.089624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76abb77-7ce5-4d5c-b82b-3fe28ebdb562 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:48.089907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76abb77-7ce5-4d5c-b82b-3fe28ebdb562 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:48.090181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76abb77-7ce5-4d5c-b82b-3fe28ebdb562 None admin] PATH_INFO: `/v3/groups/48721632e6eb4c6991278027d7f8e8bf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:48.091328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76abb77-7ce5-4d5c-b82b-3fe28ebdb562 None admin] RBAC: Authorizing `identity:get_group(group_id=48721632e6eb4c6991278027d7f8e8bf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:48.092795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76abb77-7ce5-4d5c-b82b-3fe28ebdb562 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:48.094401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 235/454] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/groups/48721632e6eb4c6991278027d7f8e8bf => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:48.100321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:48.195199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73927c9d-b74f-4699-ae46-64113962a15e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:48.227663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:48.228304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:48.228585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:48.228891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:48.229710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] RBAC: Adding query filter params (group.id=48721632e6eb4c6991278027d7f8e8bf, role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=7d83853bf6704e0fa8f738c857c951c2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:48.230018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:48.231421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190af82-e99a-4661-b508-ef016e5b80fd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:48.237055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 220/455] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Jul 18 10:22:48 2022] GET /identity/v3/role_assignments?group.id=48721632e6eb4c6991278027d7f8e8bf&scope.project.id=7d83853bf6704e0fa8f738c857c951c2&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:50.884311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502c86-c856-4660-8371-94ef3f745ac7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:50.884903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502c86-c856-4660-8371-94ef3f745ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:50.885011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502c86-c856-4660-8371-94ef3f745ac7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:50.886600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 236/456] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:50.902821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432dfa8-26e6-4397-b714-db9a2ede6758 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:50.903730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432dfa8-26e6-4397-b714-db9a2ede6758 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:50.904234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432dfa8-26e6-4397-b714-db9a2ede6758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:50.927721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2432dfa8-26e6-4397-b714-db9a2ede6758 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:50.929322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2432dfa8-26e6-4397-b714-db9a2ede6758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:50.983152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:51.023425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 221/457] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:51.063539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502c86-c856-4660-8371-94ef3f745ac7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:51.082830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c234-b0cf-4f12-a4aa-70c447cdda27 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.083167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c234-b0cf-4f12-a4aa-70c447cdda27 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.083394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c234-b0cf-4f12-a4aa-70c447cdda27 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.085136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 222/458] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:51.097844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.098139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.098420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.124964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:51.126164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:51.205543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502c86-c856-4660-8371-94ef3f745ac7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:51.278348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 223/459] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:51.288283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:51.289122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.290137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.290651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.302738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:51.305114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:51.325498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:51.343772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:51.344753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:51.345607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:51.345607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:51.345607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:51.345607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:51.352792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 237/460] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:51.355027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:51.393056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:51.504630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f2ea7-1a30-4d04-a42e-3059ca973933 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:51.506701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:51.529086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2378b-bc4e-421b-97a2-eb3044e7ff1e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:51.552814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:51.554419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.554993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.555505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.568920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:51.571106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:51.575702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:51.576622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.577119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.577605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.578519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:51.579010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:51.581138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1b9a9-d9ce-4275-a9d5-76b23f1542cd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:51.599359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:51.600267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:51.601135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:51.601135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:51.601135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:51.601135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:51.601343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 238/461] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:51.603116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01696efa-a695-49da-8958-99af63c33280 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.603410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01696efa-a695-49da-8958-99af63c33280 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.603688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01696efa-a695-49da-8958-99af63c33280 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.610096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 239/463] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:51.610096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01696efa-a695-49da-8958-99af63c33280 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:51.610643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 224/463] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:51.611707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:51.723899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:51.733376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f94466-50d3-4a60-8682-9a19c364f33e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:51.747500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01696efa-a695-49da-8958-99af63c33280 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:51.773049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:51.773996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.774452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.774886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.775780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:51.776240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:51.777814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313b211-edf0-4c16-b443-2e8866cd93bc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:51.795604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 225/464] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:51.795604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fc040-6dda-4939-96eb-a557dc45cb22 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.795604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fc040-6dda-4939-96eb-a557dc45cb22 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.795604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fc040-6dda-4939-96eb-a557dc45cb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.800372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:51.801232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.801514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.801789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.802656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:51.804504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:51.819177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4c0fc040-6dda-4939-96eb-a557dc45cb22 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:51.821240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c0fc040-6dda-4939-96eb-a557dc45cb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:51.831904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:51.904011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 240/465] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:51 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:51.904011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:51.905072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 226/466] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:51.908741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76b2f7-d8d6-42dd-b113-0f0f0b6f8685 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.908741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76b2f7-d8d6-42dd-b113-0f0f0b6f8685 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.908741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76b2f7-d8d6-42dd-b113-0f0f0b6f8685 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.909606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 227/467] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:51.920269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:51.920760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:51.921226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:51.948739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:51.950660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:52.050964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3293b4-6405-400b-a0ca-c545a118a147 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.057567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 228/468] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:52.059105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.093415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.094573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.095162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.095682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.096709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.098636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.108261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:52.151385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:52.176825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6f991-9769-4b1a-970d-74d088992623 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.196078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 241/469] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:51 2022] POST /identity/v3/users => generated 246 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:52.197952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.225471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.227637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.228078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.228487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.244224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.246911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:52.289277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:52.290707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:52.291839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:52.291839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:52.291839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:52.291839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:52.294327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 229/470] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:52.300516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.343420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606e2fa-2942-4691-ac3b-a26e1af7fd59 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.409376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.410310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.410592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.410867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.431116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.432611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6bfbab0f-0d71-47ad-ba01-b21d8ef8c095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:52.452293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:52.453250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:52.454059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:52.454059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:52.454059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:52.454059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:52.454059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 242/471] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:52.454964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6316b-1e9f-4b3c-a82d-e0be39ce6155 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.455191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6316b-1e9f-4b3c-a82d-e0be39ce6155 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.455478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6316b-1e9f-4b3c-a82d-e0be39ce6155 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.456968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 243/472] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:52.463020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6316b-1e9f-4b3c-a82d-e0be39ce6155 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.464348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7186a-8ab3-4b3a-a3ee-1694fc527f6e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.498149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:52.509256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.515852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.515852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.515852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.515852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:52.515852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.515852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b59d7-9571-4ccd-9304-825aa2ffa1b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.518261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:52.528970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 230/473] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:52.530306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.530600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.530884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.553133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:52.554472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:52.601912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6316b-1e9f-4b3c-a82d-e0be39ce6155 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.634263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.635094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.635492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.636115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.637099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:52.637698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.639425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.639802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 231/474] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:52.640789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.664198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 244/475] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:52.669649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.765587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d6512-4d1c-4355-958b-31c15b985cad None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.784421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:52.786770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f863c8-a8bb-41d6-ba68-12eb77d0b928 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:52.789328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:52.804761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.806367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.806890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.807364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.808271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.810068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.824369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:52.832863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:52.832863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:52.832863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:52.832863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:52.832863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:52.911073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 232/476] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:52 2022] POST /identity/v3/users => generated 248 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:52.912078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:52.949984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 245/477] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:52 2022] POST /identity/v3/users => generated 249 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:22:53.089183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccd47c-2411-4af1-a587-354252171e8b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:53.138562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:53.139250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:53.139536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:53.139817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:53.140588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:53.142053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:53.183233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:22:53.183233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 10:22:53.183233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [parameters: {'id': 'fa0e8be2d1c94fbcbc766b34684d8e3d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 10:22:53.183233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jul 18 10:22:53.200789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:22:53.201672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:22:53.202556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:22:53.203439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:22:53.204306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:22:53.205150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'id': 'fa0e8be2d1c94fbcbc766b34684d8e3d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:53.206007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 10:22:53.208522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:53.210752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 233/478] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Jul 18 10:22:52 2022] POST /identity/v3/roles => generated 144 bytes in 291 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:53.210888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:53.331962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42931ec6-3d3e-47ec-bf5f-42a229cf330d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:53.377467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999411c-0cd3-4a0a-a183-432e3403678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:53.378300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999411c-0cd3-4a0a-a183-432e3403678a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:53.378585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999411c-0cd3-4a0a-a183-432e3403678a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:53.378861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999411c-0cd3-4a0a-a183-432e3403678a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:53.390191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999411c-0cd3-4a0a-a183-432e3403678a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5999411c-0cd3-4a0a-a183-432e3403678a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:53.398397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:53.401179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:53.403189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 246/479] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:22:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:53.410344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:53.466339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:53.547366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c55ec-7e4c-4d31-8929-61cd597ec493 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:53.588673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:53.589368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:53.589651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:53.589925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:53.590680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:53.590984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:53.592339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab95e6-7930-4c30-91f9-96018fdab88a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:53.600989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 234/480] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:54.319052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd37f00-f71e-4539-bf28-b838d5f37762 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:54.319605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd37f00-f71e-4539-bf28-b838d5f37762 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:54.319860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd37f00-f71e-4539-bf28-b838d5f37762 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:54.321698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 247/481] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:54.327467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:54.328085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:54.328085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:54.351526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:54.352830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:54.448798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 235/482] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:54.484101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd37f00-f71e-4539-bf28-b838d5f37762 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:54.515614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:54.617754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd37f00-f71e-4539-bf28-b838d5f37762 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:54.652636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e85e6-b912-49f9-8a00-078eca636279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:54.653495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e85e6-b912-49f9-8a00-078eca636279 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:54.653791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e85e6-b912-49f9-8a00-078eca636279 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:54.654066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e85e6-b912-49f9-8a00-078eca636279 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:54.665654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e85e6-b912-49f9-8a00-078eca636279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-949e85e6-b912-49f9-8a00-078eca636279 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:54.673614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:54.675509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:54.677270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 248/483] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:22:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:54.684703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:54.735937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:54.758306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b36c-b7e3-4971-97c6-e49ad6ebbfa5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:54.758607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b36c-b7e3-4971-97c6-e49ad6ebbfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:54.758855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b36c-b7e3-4971-97c6-e49ad6ebbfa5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:54.760946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 249/484] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:54.766917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:54.767252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:54.767549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:54.805850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:54.807093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:54.826036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59164309-0fde-4b8d-85b9-10b0c3e70276 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:54.862971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:54.863972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:54.864467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:54.864964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:54.865922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:54.866420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:54.868062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:54.877262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 236/485] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:22:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:54.882921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:54.938069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 250/486] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:54.978361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:54.985754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:54.989140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720d37a-d512-4cea-963c-d561374fa6aa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.007403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:55.026427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.028462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.029149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.029824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.048260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.050562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:55.087867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:55.088773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:55.089643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:55.089643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:55.089643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:55.089643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:55.096265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 237/487] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:55.102627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.154887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc2d41-cb62-4fd6-b692-1a802e1634fc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.192109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.193389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.194154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.194638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.207624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:55.217646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:55.218540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:55.219395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:55.222793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 251/488] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:22:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:55.232719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.338592 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb1f84-10b5-434f-9121-71c739186801 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.372711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:55.408060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.409635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.410588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.410882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.411633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:55.411948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.413359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.424868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55b77-ef6d-454d-b87a-7d1a09c267a6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.439899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 238/489] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:55.445111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.448757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:55.489319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.490044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.490334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.490613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.491372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:55.491683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.494375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.513596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 252/490] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:55.514936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.621110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970f745-fe3b-43cc-a12f-27be1284e3c5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.626864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd837ece-2bce-4190-85ae-d2ebafcdd691 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.658510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:55.663507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.665051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.665431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.669076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.669186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.669186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.669186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.669186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.676357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.676838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.677837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.678310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-78fbd36e-2047-4b12-9d97-3c6d5207d962 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:22:55.695578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:55.696507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:22:55.697436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4fc0a521-d4ed-4b16-a5b8-efa1f368b5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:55.697938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:55.698673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:55.699493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:55.699493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:55.699493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:55.699493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:55.699695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 239/491] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:55.699759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 253/492] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:22:55.699759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b112846-10af-47a5-b7fb-a4601d06658a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.699871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b112846-10af-47a5-b7fb-a4601d06658a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.700823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b112846-10af-47a5-b7fb-a4601d06658a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.704808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 254/493] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:55.704936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89474321-3d10-4aaa-8bc0-ce5f0a207c07 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.705014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b112846-10af-47a5-b7fb-a4601d06658a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.705082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89474321-3d10-4aaa-8bc0-ce5f0a207c07 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.705082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89474321-3d10-4aaa-8bc0-ce5f0a207c07 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.706970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 240/494] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:55.708369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.708654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.708957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.725480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:55.727649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:55.789019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:55.801171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 241/495] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:55.802058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.817904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b112846-10af-47a5-b7fb-a4601d06658a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.857007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.857681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.857974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.858250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.859091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:55.859399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.860859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a2f3b-6991-45fd-8efc-38bde1994a5f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.902949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 255/496] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:55.902949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.902949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.902949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.918388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:55.919583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:55.921884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b646-cfad-4ff6-b8dd-3f38191dcaa9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:55.955457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:55.957937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:55.958608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:55.958896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:55.959174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:55.959907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:55.960218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:55.961641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:55.973266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 242/497] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:55.974106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:55.991369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 256/498] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:55.993024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.040496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.092196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb339b4-1565-4ed0-b3d5-a1774370dd67 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.110986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3979b-0280-4e53-a430-f998f5e40a3b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.133613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.135389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.136131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.136594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.145869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.146770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.147082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.147362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.151045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:56.162031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:56.162922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:56.163712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:56.166209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 243/499] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:56.166946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.200403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.228991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.230538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.282077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbee5-f87e-4f64-919d-02ec518b35b5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.295691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.316948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.317734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.318016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.318290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.326527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.327942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.335614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 257/500] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:56.337252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:22:56.343460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:56.344943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 244/501] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:56.346209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.409541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.438786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9a5bd-fcc5-4741-a8e9-c45a512a88c1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.446319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30364b6c-94c3-4ba4-bf23-ce2b395a6d22 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.473070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.473855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.474148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.474424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.479442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.480096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.480390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.480671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.481452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:56.481771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.483137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c6e6-89b1-4d5d-98b4-117e9bb5d734 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.484552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:22:56.490949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 245/502] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:56.492228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1275f41-898e-46be-bf70-e71986599a94 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.492553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1275f41-898e-46be-bf70-e71986599a94 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.492867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1275f41-898e-46be-bf70-e71986599a94 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:22:56.492964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:56.493774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:56.497428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 258/504] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:56.498109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 246/504] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:56.498175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.498238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1275f41-898e-46be-bf70-e71986599a94 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.536055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.598839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83708d5-43ac-4459-852d-569ca3bcfd0c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.602835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1275f41-898e-46be-bf70-e71986599a94 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.628140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.632949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.633997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.634286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.634568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.635538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.636215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.636490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.636791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.637913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:56.637913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.639381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.645403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.646862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1c77a368-37a3-444e-998e-eb3e11465c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:56.665442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:56.666453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:56.667281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:56.667281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:56.667281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:56.667281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:56.667281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 259/505] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:56.667824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.668084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.668368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.681523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 247/506] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:56.681523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.686079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:56.687282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:56.753538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 260/507] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:22:56.754285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.778191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.786608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e07fb-b8c5-4179-b678-ca4b70151e2b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.821334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.822127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.822414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.822698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.832944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.834407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:56.852193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:56.853132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:56.853982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:56.853982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:56.853982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:56.853982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:56.853982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 248/508] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:56.854248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad720-5a69-4df4-8561-4a7ebba72fb9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.854329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.871128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:56.891366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.892035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.892328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.906674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.906674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:56.906674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.906674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:56.906674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 261/509] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:56.906674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:56.961577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e620-310f-42b1-9977-6eb874600deb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:56.994574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:56.995228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:56.995514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:56.995797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:56.996519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:56.996868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:56.998243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.001402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebb065-4e82-4469-aa01-4549550b5b70 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.009782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 249/510] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:57.010609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.014081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.039602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.040268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.040557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.040870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.085056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.108067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.109548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.109783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a1454-0f1e-4669-ac7e-7d69b9f72208 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.143886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.144673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.144990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.145271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.155117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.156568 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:57.175822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:57.176746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:57.177782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:57.177782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:57.177782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:57.177782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:57.177782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 250/511] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:57.178237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.207540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 262/512] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:57.208379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.235481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.278507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f58fc-6316-43c4-8188-9e4bb444529b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.285199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.308992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7ebc2-e857-4fe8-9c0e-db38b8f72863 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.312881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.313540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.313827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.314106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.314830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:57.315140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.316497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.332821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 251/513] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:57.332821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.347959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.348785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.349092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.349372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.360026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.361544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:57.379526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:57.380503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:57.381598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:57.381598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:57.381598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:57.381598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:57.382138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 263/514] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:57.383009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.434064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27963c2a-5e4c-4871-b19a-3554566ad12f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.450334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.467905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.468699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.469011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.469296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.475904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.477864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.481437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92926-73f8-429f-93c8-20e41e13a19a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:22:57.492943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:57.493958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:57.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 252/515] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:57.495331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.511024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.523588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.523588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.523588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.523588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.523956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:57.523956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.523956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.534424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 264/516] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:57.535227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.604514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d53d03-49f7-4606-9beb-3d87f81d9643 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.634777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e132-39d3-4b2e-915b-41e3335d90b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.639768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.640560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.640885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.641170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.651852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.653347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.667788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.668583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.668894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.669171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:57.670962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:57.671940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:57.677732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:57.677732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:57.677732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:57.677732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:57.677732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 253/517] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:57.677732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.678069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.678069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:22:57.694649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:57.696340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 265/518] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:57.702395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.707750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.748111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.783745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc3f25-5fbf-4703-b460-c261b4398d65 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.802777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bcb09-58a9-4396-a99d-a441c94dd4f1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.819771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.820942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.821237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.821514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.822242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:57.822553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.823940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.835336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 254/519] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:57.836122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.838632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.839357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.839645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.839921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.840751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:57.841060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.842485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.859938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 266/520] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:57.860790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:57.926695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.938505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d988-4af9-4e0d-bf62-2b8f43f06ada None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.958261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f1147-3f31-453e-aecd-dc58b4539b4e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:57.968319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.972204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.972913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.973398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.973865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.974937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.976374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:57.992535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:57.993224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:57.993504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:57.993823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:57.994538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:57.994842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:57.996227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.007270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 267/521] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:58.008840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.088660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 255/522] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:22:57 2022] POST /identity/v3/users => generated 247 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:22:58.089459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.109423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440efd4-8281-43d9-a6a1-88955fc978c5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.146120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.146820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.147108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.147383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.148251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:58.149251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.150749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.177450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 268/523] 10.222.0.47 () {56 vars in 1139 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:58.177450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.205697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:58.214553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b2502-7aa6-4bad-98e8-17101d9b80c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.254637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.255486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.255905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.256736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.268335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.269399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:22:58.285683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:58.286713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:58.287954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 256/524] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:58.288619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.296272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:58.302303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7c82-74c0-4c5c-a7cd-f48ae1f0bf9c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.335280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.335938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.336248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.336525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.337450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aefb3626f45544ba81603a722821fce3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:58.337764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.339258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.345856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 269/525] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/role_assignments?user.id=aefb3626f45544ba81603a722821fce3&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:58.346949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.405094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1960e-bdbd-4d62-b4e7-fe63a4356608 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.461181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.462004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.462294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.462856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.473031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.473742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:58.476281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.484707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7db69-3323-4234-96a7-5681e092b6ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:22:58.496434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:58.497810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:58.499022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 257/526] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:58.499955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.533359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.534016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.534306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.534591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.575521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:58.610321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.612133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.620704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98dc51-f66f-4e20-9288-ff06b0bdf815 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.657854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.658862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.660298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.660589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.661481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:58.661801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.663355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.687728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 258/527] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:58.688997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.732709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:58.745438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 270/528] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:58.746513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.817496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204fb6e-1d56-408d-9c38-da04468c58f4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.845817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:58.858232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.859318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.859882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.860387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.861414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:58.861933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:58.863762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:58.884642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf251ef-c51b-48da-9045-87df9c9c247d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:58.885679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 259/529] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:58.886822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:58.932433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:58.933441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:58.933905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:58.934359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:58.994253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.005880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.007622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.011833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f5738-51de-4a24-a321-e0119e20c6ae None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.056326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.057440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.057735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.058020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.071341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.073045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:59.095858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:59.096792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:59.097647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:59.097647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:59.097647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:59.097647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:59.099857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 260/530] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:59.100846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.135518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 271/531] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:22:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:22:59.135518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.146607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.218122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911c209-5e05-4fad-9937-829bd0b8ccfb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.254548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.255529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.256007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.256112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.257017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.258125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aa1f5d0bba8a4178977c19b7503ef6a0) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:59.258836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.259990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891df709-4302-4134-ac99-5a9d3fcca8a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.260917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.267642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 261/532] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/role_assignments?user.id=aa1f5d0bba8a4178977c19b7503ef6a0&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:59.268828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.307743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.308427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.308739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.309024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.309764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:59.310076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.311566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.326007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 272/533] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:59.330737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.381525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.387118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517ddb5-611a-45ae-88b6-d1e5903602d2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.424871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.425960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.426392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.426833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.437987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.439700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.449965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4177da-827d-4d7e-95e0-e6d97ad9e49c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:22:59.459836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:59.461075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:59.462358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:22:59.462358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:22:59.462358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:22:59.462358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:22:59.464474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 262/534] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:59.470208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.488277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.489047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.489294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.489577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.498874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.499974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:22:59.516726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:22:59.517824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:22:59.518928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:22:59.524997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 273/535] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:22:59.531525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.569089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.606148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a6ce3-25b0-4f42-8d76-fa69744d7df7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.642305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d74e-4469-459b-ac2c-5b7e0da5159c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.670546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.671635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.672138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.672782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.673794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:59.674310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.676709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.692024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 263/536] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:59.697198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.713520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.733473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.735215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.735506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.735883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.736712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:59.737043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.738491 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.761155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 274/537] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:22:59.767416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:59.876348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:59.880453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c84522-76ba-472d-8a03-6fe33488b1c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.911102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb75efc-dc55-41a0-8c13-807015e8ab36 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:59.947518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.948522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.949034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.949514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.950635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=ce0633dadc8b43c5a13f52cace5bb85d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:22:59.951236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:59.952931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:59.961620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 275/538] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/role_assignments?user.id=ce0633dadc8b43c5a13f52cace5bb85d&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:22:59.978897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:59.979725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:59.980013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:59.980290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:59.999021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:00.000485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:00.015844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:00.016808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:00.017942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 264/539] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:00.023062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:00.184785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0d08b-01a6-47e9-a832-5541fa1b8bf8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:00.234295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:00.235334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.235855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.236335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.237359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:00.237877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:00.239553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c708510-01b2-45d2-a7a3-705952a3a420 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:00.249027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:00.268893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 276/540] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:00.275965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:00.355381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d28ed-238d-4ad5-bb3a-5de990c5c8ba None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.356026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d28ed-238d-4ad5-bb3a-5de990c5c8ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.356541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d28ed-238d-4ad5-bb3a-5de990c5c8ba None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.359516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 277/541] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:00.363845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.364359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.365173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.385059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:00.387242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:00.397808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:00.446683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843105a-6212-4329-85ae-aeb93171526d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:00.471146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 278/542] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:00.505412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:00.506113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.506364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.506651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.507497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=bd8f56a4b2d94ec1aec6f0d044cbc1bf) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:00.507874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:00.509850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:00.520760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:00.521410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 265/543] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:00 2022] GET /identity/v3/role_assignments?user.id=bd8f56a4b2d94ec1aec6f0d044cbc1bf&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:00.633401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384a53-eeaf-4012-a2d7-b2c4a376f3df None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:00.671905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab88ca-0bd9-4210-b1a9-cbbbc76b5153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:00.673162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab88ca-0bd9-4210-b1a9-cbbbc76b5153 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.673691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab88ca-0bd9-4210-b1a9-cbbbc76b5153 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.674153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab88ca-0bd9-4210-b1a9-cbbbc76b5153 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.677322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:00.687158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab88ca-0bd9-4210-b1a9-cbbbc76b5153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-eeab88ca-0bd9-4210-b1a9-cbbbc76b5153 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:00.695045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:00.696003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:00.696875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 279/544] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:00.713605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:00.751438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec31a5b-c51c-45e9-a0da-3e80ede436d9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.751764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec31a5b-c51c-45e9-a0da-3e80ede436d9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.752012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec31a5b-c51c-45e9-a0da-3e80ede436d9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.753761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 280/545] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:00.761071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.761366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.761654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.785695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:00.786939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:00.882499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595af18d-2fe4-44ab-b259-255ad0bc58c0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:00.892793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 281/546] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:00.912198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:00.982027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:00.983062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:00.983547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:00.984032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:00.985032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:00.985542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:00.987192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.000705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:01.007064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 266/547] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:01.012985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.026017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6d39-6bc6-46fa-9150-d35b1bad40d1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.042540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:01.071084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714184bd-a21c-458f-901f-867c260160cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.071886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714184bd-a21c-458f-901f-867c260160cb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.072175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714184bd-a21c-458f-901f-867c260160cb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.072457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714184bd-a21c-458f-901f-867c260160cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.099645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714184bd-a21c-458f-901f-867c260160cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-714184bd-a21c-458f-901f-867c260160cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:01.114542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:01.115451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:01.116747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 282/548] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:01.138027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714184bd-a21c-458f-901f-867c260160cb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.225681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7487abd-7b82-4a4c-914d-5e6c15d4fc3f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.289244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.290055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.290340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.290623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.311243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.312735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:01.345140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:01.346066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:01.347096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:01.347096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:01.347096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:01.347096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:01.347335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714184bd-a21c-458f-901f-867c260160cb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.347420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 267/549] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:01.355036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.423156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.423834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.424117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.424394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.425179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:01.425485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.426881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.447150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 283/550] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:01.448935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:01.454215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.518754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234532b5-ce97-4931-b122-6c94f7ea6e0c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.534438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:01.578153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.578817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.579103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.579383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.580116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:01.580425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.581828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.610904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 268/551] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:01.616383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.654472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddab713-c91e-43dd-874d-5b9598595344 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.718961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.719793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.720079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.720419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.738171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41a4fa-fb59-4f41-8e6e-0fb9b03070b1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.742660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.744138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.748767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:01.775466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.776588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.777144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.777627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:01.780898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:01.781757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:01.787451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:01.787451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:01.787451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:01.787451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:01.787451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 284/552] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:01.793162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.794229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.795400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-da4a0054-de0a-41ac-bdd9-c961c7a20033 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:01.811243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:01.812223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:01.813206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:01.816607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 269/553] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:01.818069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262534f-7087-4473-a2b1-68634accd8ff None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.818359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262534f-7087-4473-a2b1-68634accd8ff None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.818639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262534f-7087-4473-a2b1-68634accd8ff None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.823513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 270/554] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:01.824702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262534f-7087-4473-a2b1-68634accd8ff None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:01.923711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:01.941405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc77f11-4e68-4915-a5a3-f75d9567fba1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.942777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262534f-7087-4473-a2b1-68634accd8ff None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:01.985335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.986023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.986309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.986617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.987445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:01.987764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.989274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81947e17-ec30-491d-b404-e5ec3fdc4942 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:01.993913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:01.995056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:01.995550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:01.996036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:01.997158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:01.997748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:01.999466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.014204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 271/555] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:02.014204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.014204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.014204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.019054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 285/556] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:02.020202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.035615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:02.036889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:02.058896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.109062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 272/557] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:02.109909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.135046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c89871-0a2b-4ad9-ac0d-ae74520587de None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.173595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fcea2-3567-4908-9117-c3e9595792ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.174604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fcea2-3567-4908-9117-c3e9595792ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.175097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fcea2-3567-4908-9117-c3e9595792ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.175607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fcea2-3567-4908-9117-c3e9595792ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.195915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.218365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d9e00-0390-4d5d-9404-b75916fa5d77 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.257573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fcea2-3567-4908-9117-c3e9595792ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:02.259918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fcea2-3567-4908-9117-c3e9595792ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.260063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.266034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.266411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.266769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.275526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:02.277094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:02.293263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:02.294672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 273/558] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:02.295669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.317235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.374572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 286/559] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:02.376234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68e9c6-f71d-4693-9448-a8a44f2cc6a3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.376776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68e9c6-f71d-4693-9448-a8a44f2cc6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.377252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68e9c6-f71d-4693-9448-a8a44f2cc6a3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.384176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 287/560] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:02.385045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68e9c6-f71d-4693-9448-a8a44f2cc6a3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.417377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0959d-8ac8-4f6e-8a2c-2f6044f90093 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.438402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.455302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.456106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.456393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.456672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.469025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:02.477316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:02.478310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 274/561] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:02.479777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.496499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68e9c6-f71d-4693-9448-a8a44f2cc6a3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.535959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.536644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.536944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.537225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.538078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:02.538393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:02.539866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60183a1-5781-4552-82dc-24a111ae824e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.562147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 288/562] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:02.562147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.562147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.562147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.576085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:02.577302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:02.577713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.605872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79f6b-e4aa-44d2-bb86-b10fe45bdb7f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.644895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 289/563] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:02.645692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.645820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.646834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.647322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.648078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.666563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:02.669453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.681558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:02.688166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:02.698279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:02.699151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:02.699151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:02.699151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:02.699151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:02.699151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 275/564] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:02.699151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.760138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ab5af-b77c-49fa-926c-47ce4bcc2994 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.797582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.797938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f61a81-bcc0-4b8f-b5e2-f451f5cde8a4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.798354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.798624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.798916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.799853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:02.800295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:02.801728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.815137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 290/565] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:02.815137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.829443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.838024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.843868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.843868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.843868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.850253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:02.852260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:02.869834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:02.870813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:02.871737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:02.871737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:02.871737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:02.871737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:02.871737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 276/566] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:02.872815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:02.924140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:02.928421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134b787-2749-46dd-9c8b-22326b45ab8e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.963143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:02.963957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:02.964239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:02.964509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:02.972798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f8f3a-f5ca-4695-a6cd-f1fcaa755d3e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:02.974179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:02.981873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:02.982844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:02.983679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 291/567] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:02.984401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.011543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.012203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.012489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.012796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.013521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:03.013832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.015191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.026579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 277/568] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:03.027367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.083376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485fba4-d949-4120-bf89-8eea5e7a701e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.087369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.116733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.118113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.118412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.118748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.127792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5f81e-e20f-4ba3-8b2f-236d9811f642 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.129891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.131358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a2ba5c35-8571-497b-92a2-cd7055030224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:03.148192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:03.149191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:03.150127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:03.150127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:03.150127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:03.150127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:03.150127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 292/569] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:03.150985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee944c-598e-49c3-a341-377d72003a93 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.151273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee944c-598e-49c3-a341-377d72003a93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.151557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee944c-598e-49c3-a341-377d72003a93 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.152833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 293/570] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:03.153595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee944c-598e-49c3-a341-377d72003a93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.166381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.167046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.167330 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.167652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.167974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.168394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:03.168714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.173435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.181789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 278/571] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:03.182589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.258861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee944c-598e-49c3-a341-377d72003a93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.282714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.285571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799bdad-6de1-49fa-b7f9-9f471c97231e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.294182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.294862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.296248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.296248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.296248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:03.296595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.297854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.306206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 294/572] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:03.307001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.322192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.322987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.323276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.323553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.331186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.332907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-37eb7c3f-afda-4a00-8f3b-08878a591f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:03.347238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:03.348329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:03.349440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 279/573] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:03.350056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.350056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.350296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.371590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:03.372963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:03.397216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.430765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2163fe-abbf-4ff8-9891-b685f0754766 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.439468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 280/574] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:03.440308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.466554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.467215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.467498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.467774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.468492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:03.468833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.470219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.482637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 295/575] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:03.483430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.537122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.549658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed65b-d83b-4755-bd6f-05ae68753009 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.585322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.586666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.586954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.587412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.588483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fab62b-2729-4f99-8dca-8daa3dfcefa3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.594154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.595586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:03.610002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:03.611025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 281/576] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:03.611931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.623530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.628228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.629051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.629337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.629616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.640131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.641597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:03.659571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:03.660652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:03.661856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:03.661856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:03.661856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:03.661856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:03.661856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 296/577] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:03.661856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.715076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031878b8-bfdf-457b-803d-cc779d095192 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.747790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.748442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.748758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.749036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.749830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:03.750140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.751558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.761186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d17f4d-0e13-4007-a3f2-72048760efd2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.763269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 282/578] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:03.764074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.777623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.798575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.799411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.799693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.799967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.809814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:03.817534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:03.818600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:03.819427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 297/579] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:03.820720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.866388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:03.871525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a261ef-a9bf-48c2-bc52-731ae4179ec2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.908217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.909043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.909326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.909602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.916838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.918285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.923054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47124e33-f637-4a26-ae5c-1b7a52e7ce17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:03.932901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:03.934327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 283/580] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:03.935832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:03.960500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:03.961730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:03.962019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:03.962298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:03.963079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:03.963392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:03.964853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:03.976576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 298/581] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:03.977413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.004933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.035508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed167d-5fef-422f-9cc6-427b6b5f41e6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.070898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.071186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.071461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.072187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:04.072496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.073921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.074525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c6e2f-8684-4db4-98ff-61e9f8226eae None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.076802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.085329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 284/582] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:04.086803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.108595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.109315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.109604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.110238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.111124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=5554e4999dc04da682c7f1cbd5f75abc) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:04.111574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.113344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.118812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 299/583] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:03 2022] GET /identity/v3/role_assignments?user.id=5554e4999dc04da682c7f1cbd5f75abc&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:04.119535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.196762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dc650-76e5-4af4-bfc1-0c064ed2e95f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.231177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.237894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a5a4-0d52-464c-a856-62988f9b182c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.240902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.241590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.241908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.242943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.244046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:04.244417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.245853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.259813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 285/584] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:04.261117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.296925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.297723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.298161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.298268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.299112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:04.299424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.300883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.320078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 300/585] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:04.321986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.386030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.390417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a0cff-3c36-4a23-b7b3-c57dfe7cfa43 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.426867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.427916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.428204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.428483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aefb3626f45544ba81603a722821fce3/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.448945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=aefb3626f45544ba81603a722821fce3, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.450387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.463566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76945c-76ca-4c3c-addf-7f7cd180cdb7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.497668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.501614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.501614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.501614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.506597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.508745 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:04.523334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:04.524445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 301/586] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:04.525655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.535345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 286/587] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:04 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aefb3626f45544ba81603a722821fce3/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:04.536327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.645146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.656264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.673515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27ee6-cc05-4119-9801-f963ce973cd1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.683841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea55cc5-b832-4af7-bf49-799f75cea4d1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.718210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.719350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.719645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.719923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.731324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.733927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.734045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.734133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.734133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.734266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.746381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.747991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:04.754107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:04.754972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:04.755847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:04.755847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:04.755847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:04.755847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:04.755847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 302/588] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:04.756127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:04.779472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:04.780576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:04.781603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:04.781603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:04.781603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:04.781603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:04.781603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 287/589] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:04.781954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.892138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7db4ba-dd64-46b3-b392-adefbcb0376a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.892593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.902042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c28b4c-7843-4eda-9f6b-dbef591b5d79 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:04.935598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.936703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.937330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.937831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.938421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:04.938986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:04.939112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:04.939355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:04.939640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:04.939837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.940363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:04.940673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:04.941803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.942267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:04.952596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:04.956586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 288/590] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:04.962488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:04.967180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 303/591] 10.222.0.47 () {56 vars in 1139 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:04.968163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.075040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86550182-5a0a-4dc8-aed9-7d12893866ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.081506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d0262-25a4-4e98-9632-85fde1181e67 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.111776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.112904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.113440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.113966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.116797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.118378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.118628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.119006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.119836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:05.120251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.121748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.123732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.125355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.134293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 289/592] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:05.139487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:05.141017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:05.141962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:23:05.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:05.145326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 304/593] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:05.146278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.166990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:05.205429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:05.259063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7c055-5516-49a8-9feb-31e4063fc565 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.271389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eec094-6fef-463c-8a46-b9ba8039ff32 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.299470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.300543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.300873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.309112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.310264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.312458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.321487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.322586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.323088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.323549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:05.329477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:05.330587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 305/594] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:05.331980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.340638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.342507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:05.365999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:05.366944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:05.368043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:05.368043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:05.368043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:05.368043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:05.371798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 290/595] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:05.372624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.443186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:05.456386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281413b-1e51-4ab0-82c2-15e1ad57fb32 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.494047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:05.494182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba14ada-af4e-484c-8015-681e0d963a5a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.496075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.497017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.497505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.498324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.499376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:05.499830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.501452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.520462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 306/596] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:05.522004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.530386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.534583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.534583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.534583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.534583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:05.534583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.535628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.551294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 291/597] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:05.552242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.641788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057a56a-fb35-4514-8140-0796a863cf08 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.686687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.687338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.687635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.687920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.688481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4ba5-0fe1-4a6b-8949-486398c23824 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.688887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:05.689046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.690368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.708797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 307/598] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:05.708797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.723358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:05.734109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.744047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.744047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.744047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.750018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.751271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:05.752850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:05.772794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:05.774745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:05.775630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:05.775630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:05.775630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:05.775630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:05.777163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 292/599] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:05.778264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.834362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8e10e-2207-461b-b5fc-5f8f7c796656 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.881190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.882512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.883364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.883653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aa1f5d0bba8a4178977c19b7503ef6a0/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.896860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650cd36-77c7-4b12-a6d3-25aecd66d56d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:05.903085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=aa1f5d0bba8a4178977c19b7503ef6a0, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.904535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.935326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:05.937093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:05.937538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:05.937970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:05.948954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:05.950950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:05.966754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:05.967761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:05.968864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:05.975408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 293/600] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:05.975408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:05.991526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 308/601] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:05 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aa1f5d0bba8a4178977c19b7503ef6a0/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:05.998255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.022821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:06.096712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b10a9b-39d3-422d-8410-ba0492abd38d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.138714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.139405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.139691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.139967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.140739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:06.141054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:06.142454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fef746-e5d0-476d-ada1-e106d00bec37 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:06.168389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 294/602] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:06.169807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:06.169935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a31ac-9803-4de5-b28a-002e708bc7da None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.170143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a31ac-9803-4de5-b28a-002e708bc7da None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.170428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a31ac-9803-4de5-b28a-002e708bc7da None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.172018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 295/603] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:06.174181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a31ac-9803-4de5-b28a-002e708bc7da None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.184485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a98cac-0695-4b59-a43c-a88546971717 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.234756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.235368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.235661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.235947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.237106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:06.237429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:06.238929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa8972-8962-4f1a-a8e1-d61dfbe4e19a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:06.262311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 309/604] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:06.264840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.265352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.265863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.286625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:06.288170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:06.301400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:06.306290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a31ac-9803-4de5-b28a-002e708bc7da None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.345329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.346167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.346449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.346775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.354373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:06.356049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:06.371184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:06.372102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:06.373560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 296/605] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:06.374239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.376405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 310/606] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:06.378316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.488819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:06.518448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7103f8b-f7c5-460e-b5e5-3333c0df47f5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.524089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee718e8-e807-4c7b-b7bf-2a756846bc55 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.557276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.558410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.558914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.559432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/ce0633dadc8b43c5a13f52cace5bb85d/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.568311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.569030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.569320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.569603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.570994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:06.571312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:06.573054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:06.577530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=ce0633dadc8b43c5a13f52cace5bb85d, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:06.579184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:06.587815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 311/607] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:06.588702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.683721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 297/608] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:06 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/ce0633dadc8b43c5a13f52cace5bb85d/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:06.691023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.706229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb5462-951e-4dc3-b957-272ff31250ca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.712421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:06.759075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1bc0a-a9af-4976-99ba-cc705d94802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.760658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1bc0a-a9af-4976-99ba-cc705d94802c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.761015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1bc0a-a9af-4976-99ba-cc705d94802c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.761312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1bc0a-a9af-4976-99ba-cc705d94802c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.764388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:06.782350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1bc0a-a9af-4976-99ba-cc705d94802c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dad1bc0a-a9af-4976-99ba-cc705d94802c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:06.794088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:06.795122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:06.796063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:06.797180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 312/609] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:06.800704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb66d4-786e-45a8-b977-ccbc1ff1dbb2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.800704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb66d4-786e-45a8-b977-ccbc1ff1dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.800704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb66d4-786e-45a8-b977-ccbc1ff1dbb2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.804916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 313/610] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:06.806746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.807038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.807323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.829811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:06.830126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678f1a9-fddb-4c58-ac91-be9be4901580 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:06.831054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:06.872071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:06.873016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:06.873297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:06.873575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/bd8f56a4b2d94ec1aec6f0d044cbc1bf/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:06.891361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=bd8f56a4b2d94ec1aec6f0d044cbc1bf, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:06.892849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:06.916282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 314/611] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:06.917162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:06.993207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 298/612] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:06 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/bd8f56a4b2d94ec1aec6f0d044cbc1bf/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:06.994492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.072118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8ec9-a2c0-4d93-80a4-0954324db49f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.073282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.123601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbdaa1-016d-42ed-8517-aabd929b9cf7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.133619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.166935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.167753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.168038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.168316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.182035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:07.189426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.190120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:07.190238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:07.191073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:07.191921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:07.192546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.192546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.192546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:07.192546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:07.193395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 299/613] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:07.193533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:07.199312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.206793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 315/614] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:07.212715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.371986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373d7ca-ced3-4cd1-a83b-3df9de83721e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.387793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aa6dd-9dd2-410e-a88c-96445a464acf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.409414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.447464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.448139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.448433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.448726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.449488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:07.449795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:07.454813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:07.465645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.469823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.470285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.470730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.477134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 300/615] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:07.483631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.485293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:07.487193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:07.510004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:07.511118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:07.512232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:07.512232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:07.512232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:07.512232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:07.514839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 316/616] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:07.522816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.608471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0b8d7-b7d5-4b87-a23b-2a54ac31e2d7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.620708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.644521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.646269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.646269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.646269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.657353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:07.658799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:07.677341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:07.678280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:07.679159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:07.679159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:07.679159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:07.679159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:07.679159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 301/617] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:07.685685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.718974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.734329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4f389-8058-4f29-b2fa-6a0bc3b58aca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.793560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc6b7-d425-44c3-8d38-87d66cf3885a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.795210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.795891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.796178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.796461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.797249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:07.797561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:07.798930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:07.813393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 317/618] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:07.825524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.835612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.836640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.836961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.837245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.838090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:07.838430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:07.840102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:07.854346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 302/619] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:07.861133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.000254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.004528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db71d-7e9a-4b58-817c-f20fc76c98f0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.045811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.057471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda79c8-d038-427d-a850-249a3fe8d513 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.074266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.075063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.075346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.075623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.087541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.089812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:08.118165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:08.120529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:08.122863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 318/620] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:08.124235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.124235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.124235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.124235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.124810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.134211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.135628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:08.160213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:08.162975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:08.164810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 303/621] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:08.170080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.349453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8b6a9-0d51-41e5-9005-8a0e9cd290cf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.379388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6faaff-6611-48f1-b17c-ed5cd33a9224 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.393703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.394459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.394816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.395157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.396061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:08.396439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.397954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2ad2a-d020-4ae9-8503-afac74e76e72 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.417810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.418340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 319/622] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:08.418522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.418785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.419160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.419743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebd325-270c-4eb1-a77e-266e66e5ecad None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.420032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:08.420137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebd325-270c-4eb1-a77e-266e66e5ecad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.420361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.420473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebd325-270c-4eb1-a77e-266e66e5ecad None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.421815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965016b8-4046-4d4d-b084-d21ac11989f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.421940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 320/623] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:08.423369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebd325-270c-4eb1-a77e-266e66e5ecad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.447189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.449209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 304/624] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:08.450596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa318-8e42-4d00-a711-f807df212814 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.450892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa318-8e42-4d00-a711-f807df212814 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.451173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa318-8e42-4d00-a711-f807df212814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.470364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-697fa318-8e42-4d00-a711-f807df212814 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:08.471569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-697fa318-8e42-4d00-a711-f807df212814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:08.474806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.547059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 305/625] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:08.553354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa318-8e42-4d00-a711-f807df212814 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.553946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebd325-270c-4eb1-a77e-266e66e5ecad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.602922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.603982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.604487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.604876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.616519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.618238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:08.638514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:08.639618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:08.640749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:08.640749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:08.640749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:08.640749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:08.645890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 321/626] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:08.646806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.690430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa318-8e42-4d00-a711-f807df212814 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.719061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.738673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.739328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.739612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.739885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.754159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8f5e6-8a37-46ba-b883-452a440f47a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.761057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.793087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.794271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.794779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.795274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.808297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:08.816902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:08.817826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:08.818656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:08.822285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 322/627] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:08.823309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.828743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.830431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.933542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee735e4-dd95-4164-94ea-92f83ad9d311 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:08.941798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 306/628] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:08.942979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:08.966714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.974925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:08.975597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.975998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.976405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.977338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:08.977654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.979143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4051e0-ef2b-440f-b589-55f070791fb2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.986932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.992537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 323/629] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:08.993856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392886c-3500-4746-baed-cbd84ada6656 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:08.994149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392886c-3500-4746-baed-cbd84ada6656 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:08.994429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392886c-3500-4746-baed-cbd84ada6656 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:08.999564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 324/630] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:09.000336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392886c-3500-4746-baed-cbd84ada6656 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.055272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162bd38-1446-4631-9ecd-3557d09ff5b7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.090867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.091615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.091804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.092085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.092882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:09.093195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.094556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.104875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 307/631] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:09.105517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.120997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392886c-3500-4746-baed-cbd84ada6656 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.158636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.159664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.160105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.160533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.172036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.173692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f2356e5e-582f-450f-8f6d-5ab16aa5918c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:09.193654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:09.194670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:09.196176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:09.196176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:09.196176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:09.196176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:09.196176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 325/632] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:09.196176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.196549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.196618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.213764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:09.214972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:09.216288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b4f20-c554-4637-a297-9b7428d81529 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.218026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.255962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.259150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.259958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.260292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.260574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.273720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.277868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.297156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 326/633] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:09.297156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:09.297427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:09.298586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 308/634] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:09.299492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.406250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f422be-b59b-4d5b-96b7-dcf823a24c3f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.409891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c4753-ee29-45b4-9677-bacaa332133a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.443093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.443752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.444091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.444310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.445622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:09.445929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.447289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.448578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.449574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.449870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.450153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.459580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 309/635] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:09.460706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.465375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.467060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.486948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:09.487695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:09.488632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:09.489580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:09.489580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:09.489580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:09.489580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:09.489580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 327/636] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:09.489858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.518906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.564455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d696a0-cf3f-4006-917d-1de292cfa20f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.594244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1a872-12ae-4aa6-9446-37f2b588d8be None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.598687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.599349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.599639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.599922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.600739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:09.601059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.602548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.614301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 310/637] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:09.615094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.634815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.635614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.635900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.636178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.646255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:09.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:09.655253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 328/638] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:09.661047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.700952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.718945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962c630-fce0-400c-8300-5c652368dba7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.738649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.752563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.753408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.753676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.753951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.756542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524db82c-9a38-4899-9b4a-870d01c226e4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.761243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.762668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e56b45f2-544c-4410-a5e2-ba460b42cd37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:09.776727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:09.777898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:09.778823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 311/639] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:09.779357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2cf3e-8da6-4d7b-adc4-231561de7b2c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.779473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2cf3e-8da6-4d7b-adc4-231561de7b2c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.779756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2cf3e-8da6-4d7b-adc4-231561de7b2c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.781029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 312/640] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:09.781698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2cf3e-8da6-4d7b-adc4-231561de7b2c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.799065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.799734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.800026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.800306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.801069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:09.801383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.802746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.814767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 329/641] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:09.815567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.885187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2cf3e-8da6-4d7b-adc4-231561de7b2c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.913029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477fbd0-0921-4e08-9213-da1ba0378cdd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:09.918652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a6519-a596-4326-bb24-9da6f2ca7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.919530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6519-a596-4326-bb24-9da6f2ca7efc None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.919815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6519-a596-4326-bb24-9da6f2ca7efc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.920092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6519-a596-4326-bb24-9da6f2ca7efc None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/5554e4999dc04da682c7f1cbd5f75abc/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.932277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:09.935238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a6519-a596-4326-bb24-9da6f2ca7efc None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=5554e4999dc04da682c7f1cbd5f75abc, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.936672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a6519-a596-4326-bb24-9da6f2ca7efc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.947116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:09.947829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:09.948121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:09.948404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:09.949224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:09.949539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:09.950916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:09.959446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 330/642] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:09.960236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:09.963638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.028610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 313/643] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:09 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/5554e4999dc04da682c7f1cbd5f75abc/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:10.030312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.030804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.031313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.050561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:10.052171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:10.063204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591d7be-64b1-4c96-8efc-30265f56b936 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.107264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.108558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.108881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.109451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.110350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:10.110972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.113007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.131151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 331/644] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:10.132740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.145033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 314/645] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:10.152721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.215462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.258095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3559-47f8-4b1b-8314-69dbe2067140 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.266654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22489661-3f19-432b-af62-1e33e0eefe3a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.291859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.295829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.296625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.296945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.297220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.304370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.304717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.305195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.305485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.305764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.306342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.316511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.318642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:10.321142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:10.322293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 332/646] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:10.323937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:10.343174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:10.344110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:10.345025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:10.345025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:10.345025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:10.345025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:10.345025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 315/647] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:10.345313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.433088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835f42-0cc5-4964-8464-561c352e7c99 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.455629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342d7d5-8b38-4126-abe2-d801359a7504 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.457209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.473095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.473095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.473095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.473095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.473493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aefb3626f45544ba81603a722821fce3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:10.473493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.474207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.480940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 333/648] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/role_assignments?user.id=aefb3626f45544ba81603a722821fce3&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:10.481737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.513185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.514840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.515184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.515755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.532788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:10.548742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:10.549943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:10.550965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:10.557742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 316/649] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:10.558809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.614561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84bd23-f675-4027-9d99-466a483d140e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.625155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.666326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.667191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.667481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.667762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.668616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:10.668963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.670405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.673042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c716-a7a2-42ff-99d7-08c62a295940 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.692831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 334/650] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:10.693733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.719034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.719937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.720364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.720820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.721556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:10.721862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.723409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.735994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 317/651] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:10.736702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.753097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.819151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698fc7d-3372-4d34-8f26-79127b2499a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.860096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.860836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.861267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.861267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.861419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.864436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:10.864436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.864436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.868446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529f145-7515-4c48-86a0-472e4704c34d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.872801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 335/652] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:10.876933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.917005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.917699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.917990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.918762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.002890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.003785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6ae9d-01e0-4fa7-9760-839223dcb4b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.004540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.014910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.041680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.042476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.042759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.043034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.051854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.053336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:11.068606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:11.069607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:11.070772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 336/653] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:11.071456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.127086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.139439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 318/654] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:11.148993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.189659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c925736-ad1b-4e82-8de6-20e75594adcc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.238698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.239759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.240052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.240384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.258375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.260442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.269059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded476cd-1e1c-414f-b175-dd5d4010082a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:11.287440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:11.288357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:11.289794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:11.289794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:11.289794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:11.289794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:11.289794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 337/655] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:11.290274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.317799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.318489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.318780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.319063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.320617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:11.321036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.323129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.323342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.348086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 319/656] 10.222.0.47 () {56 vars in 1139 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:11.349018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.411293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d135d3-394f-4373-8911-b2ee51b38b14 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.422951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.452710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.453958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.454247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.454527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.469274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.471388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.482070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6829d-146b-4fa2-bc0a-f978038734ea None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:11.498035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:11.498976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:11.499860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:11.499860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:11.499860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:11.499860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:11.502201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 338/657] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:11.503299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.526557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.527737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.528053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.528337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.529101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:11.529414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.531331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.548467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 320/658] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:11.549347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.582845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.619705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25231-6392-47c8-b05f-57ad431dd539 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.665625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.666942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.667438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.668034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.687252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56a3e-4b33-49ee-bfc1-ef7e77fad0ee None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.704748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.732475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.733653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.734006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.734361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.735165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:11.735547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.737016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.755017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 321/659] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:11.756253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.756862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.758673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.851621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:11.889851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afb857-66fa-4743-aa3e-aaf379fcb0fa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:11.895517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 339/660] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:11.896390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:11.939820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:11.941204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:11.945600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:11.945888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:11.956181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:11.958190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:11.979291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:11.980297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:23:11.981304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:11.985966 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 322/661] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:11.986804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.019205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521fe83-3d17-44aa-a32a-b5a54ebbe3db None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.027910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.056769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.057934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.058482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.058962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.066858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.068620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:12.083505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:12.084430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:12.085454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:12.087725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 340/662] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:12.088857 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.118597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417e2c7-2b35-476a-af9a-7adf437b996b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.122933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.155743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.156535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.156870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.157147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.170469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.171934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:12.190399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:12.191304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:12.192172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:12.192172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:12.192172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:12.192172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:12.192172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 323/663] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:12.192445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.207095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d59f2-130a-4fe2-a0bc-2926ddf257d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.243173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.243852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.244135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.244435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.245305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:12.245617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.247029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22324-3961-4dfb-af86-380c27d4a0fd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.265395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 341/664] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:12.267063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42df42c-24d6-43ac-8238-4654b19d89dd None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.267356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42df42c-24d6-43ac-8238-4654b19d89dd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.267635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42df42c-24d6-43ac-8238-4654b19d89dd None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.269099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 342/665] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:12.273593 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42df42c-24d6-43ac-8238-4654b19d89dd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.306214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.322071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe64744-19df-4829-9a4f-0b6e626a1745 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.358198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.359296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.359296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.359495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.360232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:12.360543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.363639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.377202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 324/666] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:12.378356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.386209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.397175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42df42c-24d6-43ac-8238-4654b19d89dd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.433742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.434472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.434759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.435094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.435865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:12.436183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.437895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3813d-0879-44e8-a356-93f48b38ee32 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.454695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 343/667] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:12.454695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.454695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.454695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.473465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:12.474641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:12.489932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eb4b2-effd-4496-9434-4fc453d2d8c7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.526399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.527048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.527450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.527731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.543887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.545924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 344/668] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:12.551681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.611834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.613533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.640261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.662220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70704d17-8fa1-4513-a4a1-8e80b07d3276 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.699541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.700190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.700487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.700800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.701642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aa1f5d0bba8a4178977c19b7503ef6a0) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:12.701960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.703400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.709567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 345/669] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/role_assignments?user.id=aa1f5d0bba8a4178977c19b7503ef6a0&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:12.711081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.733335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 325/670] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:12.734519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.789728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.827959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946c46c-12db-41f2-9c5c-5daaeda76926 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.850413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41526291-a56c-4e82-a41a-c318caaa1a30 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:12.869199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.869739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.869942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.870619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.878449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:12.879906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:12.891882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:12.892780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.893074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.893352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-56bf00b2-01f4-4f66-a942-23e2e3aec460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:12.894448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:12.896511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 346/671] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ea8e-2003-4398-9d65-23aa31d3f622 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.897805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ea8e-2003-4398-9d65-23aa31d3f622 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.899241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ea8e-2003-4398-9d65-23aa31d3f622 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.903415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 347/672] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:12.904236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ea8e-2003-4398-9d65-23aa31d3f622 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.905567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:12.908516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:12.915197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:12.916192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 326/673] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:12.923432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.020597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ea8e-2003-4398-9d65-23aa31d3f622 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.030958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eee784-716d-4ffa-8322-383b4039950f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.033733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.056525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.057543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.057759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.058017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.065577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.066227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.066508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.066792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.067575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:13.067906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.069193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.069420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762513f-2f79-4904-bf1c-5a5bc539fa2a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.071150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.089028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 327/674] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:13.089028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.089028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.089028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:13.090320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:13.091176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:13.092054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:13.092054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:13.092054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:13.092054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:13.092054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 348/675] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:13.092792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.110411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:13.111587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:13.153760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.188758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 328/676] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:13.188758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.209161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d5fbb-709c-4f59-a6d0-d2ba4bd554c1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.246086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.246786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.247061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.247341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.248108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:13.248421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.249838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.259119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 349/677] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:13.259947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.309687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.318015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d9f04-32d6-4999-9e67-8e3f6c31c505 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.362325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.363071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.363255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.363537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.364369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=ce0633dadc8b43c5a13f52cace5bb85d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:13.364711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.366137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.373078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbff5e-c4fb-4eb5-b0f3-2b6938cbb452 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.381059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 329/678] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/role_assignments?user.id=ce0633dadc8b43c5a13f52cace5bb85d&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:13.381059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.407667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.422815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.423505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.430050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.430050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.430050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:13.430050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.430050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.449422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 350/679] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:13.449422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.521316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750e65e-6e44-4543-9e52-141d50c8fa32 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.575532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.576340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.576623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.576968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.590367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.592098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.601810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4674d2-0d40-4a76-9839-c53d2d0c2a2b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:13.610526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:13.611425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:13.612902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:13.612902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:13.612902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:13.612902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:13.612902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 330/680] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:13.612902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.625729 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.651583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.652389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.652671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.652985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.665034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.666656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:13.685693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:13.686652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:13.688440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:13.688440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:13.688440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:13.688440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:13.688440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 351/681] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:13.692608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.731156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c368937-cb18-4d27-880d-9b080433472b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.749129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.769399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.770189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.770468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.770742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.777878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.779457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:13.796302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:13.798380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:13.800562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:13.808720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 331/682] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:13.808720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.817883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069804de-164c-4aa0-82d4-b3195f3921e6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.858203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.858973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.859268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.859552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.860391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:13.860742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.862127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.875822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 352/683] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:13.876520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.883071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.917777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9f2e-0d44-4cf1-bb59-e695b38b0211 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.955799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.956856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.957605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.958102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.959051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:13.959568 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.962007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.979986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 332/684] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:13.980163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.981563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:13.989198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aba380-199f-4515-8619-4f6b147052f7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:14.027454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:14.028080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.028379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.028664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.029506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:14.029825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:14.031384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:14.049261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 353/685] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:14.049261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:14.105533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762b611-f19d-4b61-aaf3-d7325bae2d6d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:14.137577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:14.144549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:14.145488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.145933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.154130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.154130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:14.154130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:14.154358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41284-566a-4b84-abc3-0fd89f32d069 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:14.189714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:14.190439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.190726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.191003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.199614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:14.201114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:14.215587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:14.216521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 354/686] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:14.222535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:14.235333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:14.273617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 333/687] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:23:13 2022] POST /identity/v3/users => generated 254 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:14.274804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:14.395305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640740a-857f-46b4-a5b5-813af36ed2bb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:14.455516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:14.456523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.458114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.458720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.460054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=bd8f56a4b2d94ec1aec6f0d044cbc1bf) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:14.461707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:14.463435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:14.475053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 355/688] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:14 2022] GET /identity/v3/role_assignments?user.id=bd8f56a4b2d94ec1aec6f0d044cbc1bf&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:14.487588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761b586-eaee-48e1-bf57-c41f45cca689 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:14.527046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:14.527988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.528458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.528914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.529930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:14.530417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:14.532033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead85b8c-eb13-41b4-a82e-03d025d38866 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:14.555998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 334/689] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Jul 18 10:23:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:14.561484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:14.649087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:14.676557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4431-f6eb-4562-a872-83edfb890a62 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:14.717527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:14.718424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.718898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.719181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.798342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:14.800043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:14.892320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:14.899630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47dc1-fb05-4b74-a28e-9143bd5a729a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.900189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47dc1-fb05-4b74-a28e-9143bd5a729a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.900637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47dc1-fb05-4b74-a28e-9143bd5a729a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.903513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 335/690] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:14.916721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:14.917672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:14.918317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:14.926825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 356/691] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:14.933779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:14.939447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:14.940961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:14.990577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.024493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 336/692] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:23:15.070811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.103082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505e2a6-ae78-45a8-a7d8-6c083259d260 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.143877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.145288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.146555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.147704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.167911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.169849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:15.205050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:15.205980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:15.206859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:15.206859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:15.206859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:15.206859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:15.206859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 357/693] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:15.208305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.255602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9df71-53f4-473b-999c-d16fbfb4bb7a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.286225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.293631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.294674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.297320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.297320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.306635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:15.314662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:15.315660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:15.316501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:15.320349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 337/694] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:15.326169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.405544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.414673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3781df3-1356-46f1-8fd8-c31c01b3a36c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.481438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.482172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.482464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.482753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.483512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:15.483810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.485302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.503124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209faa3d-7058-4e06-986b-8fa73109fae5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.510140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 358/695] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:15.517664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.539867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.540935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.541447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.541922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.542911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:15.543422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.545013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.554758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 338/696] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:15.564650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.664483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.685863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12c1d3-da41-4e10-babb-721fffe9c5d4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.709518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.716631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2f802-cef3-45ab-809a-7248ea150341 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.722576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.723376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.723667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.723948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.731758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.733249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bc8580f1-ae11-4013-9dfa-d83c7734bf07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:15.747318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:15.748203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:15.749127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 359/697] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:15.750070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e076-4210-4530-a80f-2f0afa0ad4b4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.750361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e076-4210-4530-a80f-2f0afa0ad4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.750645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e076-4210-4530-a80f-2f0afa0ad4b4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.752246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 360/698] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:15.756889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.757029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2e076-4210-4530-a80f-2f0afa0ad4b4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.758218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.758505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.758786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.771058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.772620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-23869d7d-7b5c-4288-8d59-50a67944040f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:15.793435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:15.795502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:15.797702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:15.797702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:15.797702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:15.797702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:15.800479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 339/699] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:15.804721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.804721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.804721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.820366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:15.822017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:15.865090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2e076-4210-4530-a80f-2f0afa0ad4b4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.900559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 340/700] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:15.901862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.929637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.930303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.930595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.930878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.931689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:15.932010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.933495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.951677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 361/701] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:15.953105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.969104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.995140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:16.022042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087fb5-6adf-4b29-aff2-f4d1310fca4a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.060817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.061713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.062098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.062659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.063379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:16.063793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.065305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.078773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 341/702] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:16.079773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:16.096822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525604-113e-499d-aaf8-41a6dad857f7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.142974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.144410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.145114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.145551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.146630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.149478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:16.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:16.168705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:16.169791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:16.173017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 362/703] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:16.174127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:16.203828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d828b0-cf27-449f-9293-4abdf5a97bd1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.252211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:16.256020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.256878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.257519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.257985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.259218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=5554e4999dc04da682c7f1cbd5f75abc) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:16.259686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.261298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.275710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 342/704] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:16 2022] GET /identity/v3/role_assignments?user.id=5554e4999dc04da682c7f1cbd5f75abc&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:16.275710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:16.275710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:16.298216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fb8f4-1234-4c76-ad31-bb447f0192f8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.397435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06429e64-86a4-475c-a18e-489340409ae1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.406250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.407071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.407360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.407641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.434105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.435579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.442557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.443635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.447767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.448251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.449226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:16.449688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.451247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.457689 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 343/705] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:16.465724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:16.465929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:16.466817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:16.467770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:16.471900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 363/706] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:23:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:16.472862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:16.549320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:16.592350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805a0db-8ced-4078-8bfd-9fa1b75f0398 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.604185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668478b3-da31-4687-b692-55deb75b3845 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.629738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.630667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.631070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.631451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.632848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] RBAC: Adding query filter params (type=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:16.633271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.634908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.641023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:16.642893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 344/707] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:16.649396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.650341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.650781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.651239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.654868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:16.654868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.654868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0f848-5243-4b87-95cb-093b48ae7a5d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.681037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 364/708] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:23:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:16.682896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:16.801447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0e6b-5322-4a8a-9187-3603016f2663 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:16.835105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:16.842950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:16.843599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.843933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.844216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.915904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c91d4e-e71d-43d0-9593-64522b367df4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.916210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c91d4e-e71d-43d0-9593-64522b367df4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.916472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c91d4e-e71d-43d0-9593-64522b367df4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.918211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 365/709] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:16.919491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:16.921004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:16.923525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27779d-b608-4ff7-8fb9-cae208454430 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:16.923798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27779d-b608-4ff7-8fb9-cae208454430 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:16.924081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27779d-b608-4ff7-8fb9-cae208454430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:16.941252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-be27779d-b608-4ff7-8fb9-cae208454430 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:16.942429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be27779d-b608-4ff7-8fb9-cae208454430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:17.014557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 366/710] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:23:17.015908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97709-8946-485f-bd6c-68a574687cbb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.016210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97709-8946-485f-bd6c-68a574687cbb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.016497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97709-8946-485f-bd6c-68a574687cbb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.026395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 367/711] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:17.031520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.031818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.032105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.038463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 345/712] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:17.044105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.063797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:17.064957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:17.083031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.137227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 368/713] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:23:17.138181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.171832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735790-a9bf-4133-8b07-fb528254840f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.215662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.219215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.220184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.220474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.221578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.234836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.236747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:17.257236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:17.259395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:17.261506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:17.261506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:17.261506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:17.261506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:17.263427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 346/714] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:17.264267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.270669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb9c19-7555-44e4-9908-dfe938976546 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.307758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.309004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.309569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.310092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.311077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.312638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:17.327546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:17.328472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:17.329763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:17.333144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 369/715] 10.222.0.47 () {56 vars in 1129 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:17.334229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.353711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.373084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb99ac-755e-40a1-9b3e-41c5c7b142c5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.410269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.410894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.411184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.411462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.424215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:17.434527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:17.435504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:17.436321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 347/716] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:17.437060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.452400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ac74-83c1-432c-84fb-4a00bf56e4b7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.467260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.490812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.492006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.492658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.493168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.494202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:17.494716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.496476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.513704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 370/717] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:17.514883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.552007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc5695-7a9b-47d4-bb92-478747ca83cb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.592946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.593531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.593819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.594101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.594921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:17.595238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.596717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.602674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 348/718] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:17.603206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.618054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.630784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e2fe-8678-41d8-aa4f-3639a1421f61 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.667528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.668504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.668997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.669514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.670860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:17.671370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.672987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.682847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 371/719] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:17.683838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.723193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.734257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028b0f-254c-499f-85c2-d5bf623049c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.787260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.788426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.789452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.789936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.800201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.801597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca63485-4ece-49dc-b37b-de1ed5878695 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:17.802191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:17.826174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:17.827119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:17.827981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:17.830954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 349/720] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:17.831974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.841903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:17.842698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:17.843141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:17.843486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:17.844410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:17.844880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:17.846530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:17.856980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 372/721] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:17.858029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:17.918371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:17.974828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596da00c-ff1d-4a27-bdf9-6ea3096d6a77 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.055776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.057479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.057974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.058452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.076404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.077896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.087129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:18.092926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed3ef-766d-4f2f-a4fe-58a2f0b0778e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:18.100734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:18.101921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:18.102794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:18.102794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:18.102794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:18.102794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:18.102794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 350/722] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:18.106519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:18.142739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.143414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.143704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.143988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.144828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:18.145147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.146586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.160318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 373/723] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:18.166335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:18.233012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc536fa0-ed5f-4119-92b8-d2669f764409 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.278110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:18.307679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978f341-c3d3-438b-acfc-15e4cf11138b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.326897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.327580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.327872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.328232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.329017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:18.329336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.330735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.351867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 351/724] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:18.358153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:18.370448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.371110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.371398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.371683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.372523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aa1f5d0bba8a4178977c19b7503ef6a0) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:18.372888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.375659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.384815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 374/725] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:18 2022] GET /identity/v3/role_assignments?user.id=aa1f5d0bba8a4178977c19b7503ef6a0&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:18.543121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312489bf-8604-4aa4-9e8a-86ea2f86d2f9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.594800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.595639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.595922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.596198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.605766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.607156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:18.623516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:18.624517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:18.625518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 352/726] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:23:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:18.634185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:18.664201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:18.761060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6262a-d218-49c3-a8cd-9b36f1348688 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.805453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.806314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.806610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.806896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.807747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:18.808065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.809593 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bfc23-9883-46ae-82f1-c4b3a5eb29dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.835430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 375/727] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Jul 18 10:23:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:18.840786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:18.899417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b687-6aa6-489e-b67b-535548027ff2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.899609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b687-6aa6-489e-b67b-535548027ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.899905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b687-6aa6-489e-b67b-535548027ff2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.901620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 376/728] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:18.906887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.907432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.907940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.920202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:18.927954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:18.930071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:19.008172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 377/729] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:19.051015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:19.057803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd264c-2e25-45b1-8d0e-c5346b4190d9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:19.071051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:19.106879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005365e-6dcf-4516-b8c0-a3c42d01d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:19.108150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005365e-6dcf-4516-b8c0-a3c42d01d406 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.109161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005365e-6dcf-4516-b8c0-a3c42d01d406 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.109672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005365e-6dcf-4516-b8c0-a3c42d01d406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.168088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc332-1dbd-457e-b671-9d8fcfdd8a3e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:19.192542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005365e-6dcf-4516-b8c0-a3c42d01d406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:19.195379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005365e-6dcf-4516-b8c0-a3c42d01d406 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:19.211293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9fca7-6696-4f8e-b190-4b437e881ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:19.211293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9fca7-6696-4f8e-b190-4b437e881ac2 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.211293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9fca7-6696-4f8e-b190-4b437e881ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.211293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9fca7-6696-4f8e-b190-4b437e881ac2 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.211749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9fca7-6696-4f8e-b190-4b437e881ac2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:19.211749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9fca7-6696-4f8e-b190-4b437e881ac2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:19.241091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:19.315074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 378/730] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:19 2022] POST /identity/v3/services => generated 240 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:19.322862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dc154-4578-4cca-80b5-07128a3c643b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.322862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dc154-4578-4cca-80b5-07128a3c643b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.322862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dc154-4578-4cca-80b5-07128a3c643b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.322862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 379/732] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:19.323327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 353/732] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:23:19.323415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dc154-4578-4cca-80b5-07128a3c643b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:19.327760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.328109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.328395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.347031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:19.348278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:19.424969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:19.476796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dc154-4578-4cca-80b5-07128a3c643b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:19.484717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 354/733] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:19 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:23:19.529930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:19.533259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:19.534481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.534481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.534681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.546497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:19.548037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:19.567757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:19.568752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:19.570576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:19.570576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:19.570576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:19.570576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:19.570940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 380/734] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:19.575038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:19.621309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:19.658025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b453b40-55ee-4c81-a4f8-758175442d5d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:19.703187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:19.704261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.704801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.705224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.706569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e261c59-67be-4291-a1fd-8621df8f5a0b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:19.719630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:19.728447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:19.729650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:19.730498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:19.731131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:19.733368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 355/735] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:19.739372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:19.777418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:19.778219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.778581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.778962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.779772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:19.780156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:19.781657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:19.803180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 381/736] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:19.808560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:19.859108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc90e8f-869d-46cc-bdad-619814c359e8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:19.890973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:19.902147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:19.903453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:19.903905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:19.904346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:19.905464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:19.906145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:19.907761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:19.918810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 356/737] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:19.924865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.043790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f841f-f484-49e9-a1aa-dcf53a9a206d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.069223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f450f-e956-40c7-841e-ebd5594c6aa8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.093521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.094986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.095455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.095909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.109518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.111520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:20.131360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:20.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:20.134054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:20.134054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:20.134054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:20.134054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:20.136030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 357/738] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:20.142271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.143057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.152418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.152901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.153348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.166168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.167818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:20.191119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:20.192657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:20.193766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:20.196894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 382/739] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:20.202473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.232254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.249875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.352509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51fdc-eb55-4e78-af7d-28993d14cef5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.374700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68783b-2e08-4235-bb74-cbeb7913f0b4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.390292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.390970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.391257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.391538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.392329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:20.392640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.394207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.407386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 383/740] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:20.414234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.433171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.433848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.434133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.434411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.435143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:20.435452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.436919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243c03c-e7d9-4024-87e5-52917af19556 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.452148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 358/741] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:20.458263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd730adc-cd10-4ad5-96c8-51ae518622cf None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.458554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd730adc-cd10-4ad5-96c8-51ae518622cf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.458831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd730adc-cd10-4ad5-96c8-51ae518622cf None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.460285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 359/742] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:20.460883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd730adc-cd10-4ad5-96c8-51ae518622cf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.544531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.548672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029628a9-a50a-47f2-a10c-dcd5c97dffd3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.580368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd730adc-cd10-4ad5-96c8-51ae518622cf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.581960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.595016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.595693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.595984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.596310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.597364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=8721bfc5b53a421d99d1e75f84e639b3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:20.597692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.599517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9d153-e60e-4c3e-b8c1-a0752ccf15db None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.607927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 384/743] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/role_assignments?user.id=8721bfc5b53a421d99d1e75f84e639b3&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:20.609644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.610175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.610467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.626591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.628078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.628565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.629068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.631167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:20.632610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:20.642009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.644619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:20.665960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:20.666973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:20.667913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:20.670685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 360/744] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:20.677185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.715907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 385/745] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:20 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:23:20.750272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.786606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf66b0-1c57-4ae9-8952-fcae71fe7dd9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:20.825988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:20.827008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.827440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.827885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.828926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:20.829415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:20.831679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:20.853833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 361/746] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:20.859601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.908461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.963604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.979055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c47f6-1b1d-41f2-8d89-45e9e601f532 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.037738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a0bf4-5e22-4743-ad38-ee3bd7fca4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.038658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0bf4-5e22-4743-ad38-ee3bd7fca4d9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.039052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0bf4-5e22-4743-ad38-ee3bd7fca4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.039331 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0bf4-5e22-4743-ad38-ee3bd7fca4d9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.040062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a0bf4-5e22-4743-ad38-ee3bd7fca4d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.041474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a0bf4-5e22-4743-ad38-ee3bd7fca4d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.043786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c306d7-aa8b-415f-b2ff-5c339f7a1f59 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.079335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.080005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.080322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.080605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.149307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.150718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.160601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 386/747] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:20 2022] POST /identity/v3/services => generated 251 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:21.162028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194f62-da85-435a-8e29-140825d60d46 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.162328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194f62-da85-435a-8e29-140825d60d46 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.162610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194f62-da85-435a-8e29-140825d60d46 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.165130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 387/748] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:21.170498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.171462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.171919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.188720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:21.194975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:21.196557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:21.292432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 388/749] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:21.327076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:21.356444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 362/750] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:23:21.362696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:21.421137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:21.442134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe7608-d585-4529-97b9-343f743a2fde None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.471169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c6f88-1944-489e-b06c-11272ff072d4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.482229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.483567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.483858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.484140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.497818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:21.497993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:21.510662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:21.512396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:21.513270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:21.516514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.516514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.516514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.516514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.517205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 389/751] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:21.518379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:21.528552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.530357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:21.557223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:21.559173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:21.560927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:21.560927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:21.560927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:21.560927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:21.560927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 363/752] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:21.565781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:21.652437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302ed53-423f-4831-8f5f-4f50cffe02e2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.690753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.691347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.691636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.691917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.692672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:21.693061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.695160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.701151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:21.706426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 390/753] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:21.713753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:21.784304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145680-d234-4620-aa0e-545aa6820e04 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.855872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.856840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.857941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.858974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.859857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:21.860469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.862093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.871661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c217996-46c8-4f46-ac99-d05881c3a641 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.885693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 364/754] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:21.892114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:21.913969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.916172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.916791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.917076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.930967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.933575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.938793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6c9180f3-27ec-4cce-907e-a9dd807f2e1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:21.958144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:21.959062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:21.961224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:21.961224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:21.961224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:21.961224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:21.961224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 391/755] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:21.961224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed3771-ae86-4f65-a3c3-737e1706196a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.961676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed3771-ae86-4f65-a3c3-737e1706196a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.961833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed3771-ae86-4f65-a3c3-737e1706196a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.965125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 392/756] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:21.966057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed3771-ae86-4f65-a3c3-737e1706196a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.030530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.036937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facf0c1-f2dd-49b1-a0f5-ddbeef03a469 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.080785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed3771-ae86-4f65-a3c3-737e1706196a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.081403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.082528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.083773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.084125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.094066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.095904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4c09f56b-941b-460f-9719-4dba045c682b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:22.111961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:22.113074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:22.114044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:22.114734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 365/757] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:22.119060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.119060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.119060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.120666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.121370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.121664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.122072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.122923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:22.123240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.124609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.134682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:22.136218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:22.141841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 393/758] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:22.142746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.224914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 366/759] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:22.226287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.274992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.286576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7639e-ffa1-4282-b5e2-9ad4180770fe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.303962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.328708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.329703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.330165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.330594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.331673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:22.332166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.334129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.348240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 394/760] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:22.349471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.363496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90ba13-8f75-4177-a305-5a28d81e60e3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.408867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.409663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.410265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.410721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.420354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.421683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:22.436480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:22.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:22.438712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:22.442503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 367/761] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:22.443360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.489278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578dba5-af01-4156-93aa-6f8e7fc26683 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.526290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.527093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.527379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.527660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] PATH_INFO: `/v3/services/network` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.528320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.529730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:22.542267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:22.543469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:22.544455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:22.548518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 395/762] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:22.549573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.549861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.570349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3385dbd-3d69-4a60-abc4-2c313532da1d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.587448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.609259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.610162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.610625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.611073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.612107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=5554e4999dc04da682c7f1cbd5f75abc) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:22.612612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.614206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.621321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 368/763] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/role_assignments?user.id=5554e4999dc04da682c7f1cbd5f75abc&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:22.622416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.676878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9453e-5df8-4d9e-8eb6-67ffe137ea70 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.713669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.714603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.714603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.714901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.715747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:22.716059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.717659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.738311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 396/764] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:22.744576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.838058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f19553-fb85-4a6a-bb6e-b6c6ca188a8a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.883673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.884342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.884630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.884963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.885735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:22.886598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.887977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:22.893421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.895644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 369/765] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:22.901804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:22.914336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7efde-a637-48cb-aa22-8d28192e31bf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:22.916808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:22.979891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:22.980899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:22.981287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:22.981776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:22.993695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:22.995339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.026263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3f3f-a52e-41f9-bc58-7f843818d3c6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.066054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.067056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.067521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.067980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.068957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] RBAC: Adding query filter params (type=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:23.069540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.071757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d04751-ba42-418a-9b14-bfafa0ad522e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.078557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 370/766] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:23.085164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f7586-a8c8-4824-9168-1fb7b34fa93f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.085978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f7586-a8c8-4824-9168-1fb7b34fa93f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.086442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f7586-a8c8-4824-9168-1fb7b34fa93f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.088470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 371/767] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:23.089500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f7586-a8c8-4824-9168-1fb7b34fa93f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f2b940a0-078a-4a9d-9b5d-4a31bf384a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:23.138885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:23.139843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:23.140840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:23.140840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:23.140840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:23.140840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:23.144926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 397/768] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:23.146437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.146813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.147266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.164965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:23.166498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:23.175355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.211887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f7586-a8c8-4824-9168-1fb7b34fa93f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.251205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 398/769] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:23.251205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.272405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.273682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.274150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.274670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.276181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] RBAC: Adding query filter params (interface=public, service_id=44aadd504c2e4897b5b89fd149c520b9, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:23.276707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.278746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.292199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 372/770] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/endpoints?service_id=44aadd504c2e4897b5b89fd149c520b9&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:23.298439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.356120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.380255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41f994-b461-4b73-866d-9dbf4a368157 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.418178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.418995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.419342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.419751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.420532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:23.421132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.422392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.434865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 399/771] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:23.439756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3660-5d13-4a8b-bca3-f143cf4d7482 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.440564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.477942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.478918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.479187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.479588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.487153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.492132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:23.500590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:23.507213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 373/772] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:23.508053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.567777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcd146-b21e-4229-afe1-d803c241e14e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.606252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.606928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.607258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.607548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.608237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.609703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.612439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.621198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 400/773] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:23.622124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.642632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3777a16-5bed-4e6b-8ef2-b7c7a0ebd7b0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.688022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.689224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.689739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.690754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.700457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.702265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9925cc2f-57e9-42cd-b4b9-baf13961b643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:23.721571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:23.722555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:23.723476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:23.726614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 374/774] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:23.731648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186b5a8-2487-4683-9a29-1388ac681d48 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.731648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186b5a8-2487-4683-9a29-1388ac681d48 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.731648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186b5a8-2487-4683-9a29-1388ac681d48 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.731648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 375/775] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:23.733221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186b5a8-2487-4683-9a29-1388ac681d48 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.796785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a489f4-27c6-4c4f-92fe-a4ae4dde99d6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.820753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.838599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.840080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.840080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.840080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.840671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:23.841008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.843839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e725f32-7b06-4900-9aaf-bd8cefce87a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.858793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 401/776] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:23.860131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.860427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.863590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.873162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186b5a8-2487-4683-9a29-1388ac681d48 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.882535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:23.884176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:23.919424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.920339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.920802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.921248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.922210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:23.922712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.924313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.940286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 376/777] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:23.941472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.963701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.984707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 402/778] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:23.984707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.092719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f547f45-1d0d-4a3a-8994-0813a1cdcd8c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.098775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612212f9-c10d-4855-a592-74f8fce4b424 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.112668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:24.130835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.132017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.132512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.133035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.143759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.144079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.144980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.145271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.145547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aa1f5d0bba8a4178977c19b7503ef6a0/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.146571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.147808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=aa1f5d0bba8a4178977c19b7503ef6a0, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.149614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:24.170580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:24.171609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:24.172515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:24.172515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:24.172515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:24.172515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:24.172515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 377/779] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:24.172515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.257841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 403/780] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:23 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aa1f5d0bba8a4178977c19b7503ef6a0/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:24.259086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.323106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76714f0b-616a-4fe0-901b-083671e08be1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.334525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:24.396795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543061a-c69f-4776-aced-6ed9e818d83e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.406861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.407665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.408027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.408384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.409222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.410712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:24.438672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:24.439684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:24.440516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 378/781] 10.222.0.47 () {56 vars in 1129 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:24.445608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.446352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.446645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.446926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.447670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:24.447984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.449410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.453127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.469769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 404/782] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:24.475708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.498789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:24.612257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d96626-a739-483b-92fa-d639d9aa89bb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.651707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47c99b-db0d-4f5c-ad36-52596520f9e9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.675561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:24.683060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.683714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.684008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.690201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.690201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:24.690201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.690201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.700276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 379/783] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:24.706037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.716128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.717036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.717449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.717740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.731182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.740711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:24.763409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:24.764389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:24.765362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:24.768089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 405/784] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:24.780715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.860052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:24.866077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94540c-b668-457e-b49e-736e267d02ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.906085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.907086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.907675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.908212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.909238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:24.909922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.911739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a40fbb-cc05-48e2-ba87-be6872b58e45 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.917379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61dc1a-79eb-4992-be9c-d760291d992c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:24.917900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 380/785] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:24.919518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6816f3d-c01c-48c5-bafe-1897a48a7ab8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.920065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6816f3d-c01c-48c5-bafe-1897a48a7ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.920546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6816f3d-c01c-48c5-bafe-1897a48a7ab8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.922329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 381/786] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:24.923249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6816f3d-c01c-48c5-bafe-1897a48a7ab8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:24.957665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:24.958338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.958624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.958907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:24.959743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:24.960703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:24.962285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94bbd-8d21-4256-b1e8-6130ce5c4883 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:24.984622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 406/787] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:24.984622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:24.984622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:24.984622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.000351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:25.001610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:25.027404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.044443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6816f3d-c01c-48c5-bafe-1897a48a7ab8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.080811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 407/788] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:25.081700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.082542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.083494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.084009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.084536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.085948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] RBAC: Adding query filter params (interface=public, service_id=d4e4eff18e474aa587bb6b4ac4c79679, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:25.086492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.088057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.094238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 382/789] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:24 2022] GET /identity/v3/endpoints?service_id=d4e4eff18e474aa587bb6b4ac4c79679&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:25.104702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.173169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.195372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b736fa-e6e4-4f94-85d4-3b23a442d7cd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.239402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.241845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.242290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.242987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.246751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8389f-380f-4207-a12c-909ccbe406b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.261090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.264368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:25.286366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:25.287247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:25.288241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:25.288241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:25.288241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:25.288241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:25.288469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.288469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.289070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.289238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.289900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.291314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 408/790] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:25.291496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.292475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.303341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 383/791] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:25.304964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.312260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.404966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8cf25-c2b0-4910-a7b6-e00f204ecf09 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.419335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.430098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c861af5-1e89-40a1-ac06-f3be3d1cf3d3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.444462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.445685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.446170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.446670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.458648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:25.467379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:25.468369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:25.469253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:25.472148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 409/792] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:25.476125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.477174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.478300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.479262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.483984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.485536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:25.492073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.493859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.514883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 384/793] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:25.520583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.568032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.603575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0e76c-2981-44df-ba61-837790c9d42c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.650707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.652493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.654219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.654558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.656230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:25.656553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.657989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.668249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 410/794] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:25.673858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.674994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c91805-6d6b-4ece-8441-98365bf66ffe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.719693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.721062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.721938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.722739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.734028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.737723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:25.775604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:25.776546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:25.777449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:25.780070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 385/795] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:25.785855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.813063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.832728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976ed3-a53b-46e1-a1b6-ea771cb58a29 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:25.873747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:25.874617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:25.874908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:25.875194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:25.890341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:25.892628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:25.912944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:25.913948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:25.914877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:25.914877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:25.914877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:25.914877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:25.914877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 411/796] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:25.917000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:25.919863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:25.989215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64264-6a6d-4d60-832a-6cadbb3693a9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:26.088554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:26.089284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.089570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.089853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.090652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:26.090962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:26.092861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf83d92-5a4a-463a-8a6d-f5092308c77c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:26.103273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc3476-b211-4284-8224-71ba37646c3a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:26.114880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 386/797] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:26.117236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f0352-1939-46af-9b28-930fe5630370 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.117732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f0352-1939-46af-9b28-930fe5630370 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.118225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f0352-1939-46af-9b28-930fe5630370 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.119886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 387/798] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:26.120850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f0352-1939-46af-9b28-930fe5630370 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:26.150202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:26.151484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.152236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.153224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.154440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:26.155199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:26.157501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dd068-5337-4ed3-9a68-34a4c187a6f1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:26.176093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 412/799] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:26.178291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057522f-6331-4dce-98a3-7007e917669d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.179303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057522f-6331-4dce-98a3-7007e917669d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.180347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057522f-6331-4dce-98a3-7007e917669d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.203113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e057522f-6331-4dce-98a3-7007e917669d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:26.204920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e057522f-6331-4dce-98a3-7007e917669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:26.209620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:26.277444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f0352-1939-46af-9b28-930fe5630370 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:26.299847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:26.352331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:26.354055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 413/800] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:26.355332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.356081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.357009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/8721bfc5b53a421d99d1e75f84e639b3/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.357939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057522f-6331-4dce-98a3-7007e917669d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:26.387513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=8721bfc5b53a421d99d1e75f84e639b3, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:26.390195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:26.477659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 388/801] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:26 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/8721bfc5b53a421d99d1e75f84e639b3/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:26.479416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:26.534519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057522f-6331-4dce-98a3-7007e917669d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:26.617654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:26.620916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.621448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.622977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.651353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:26.653216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:26.674492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b368f-2e95-4b27-8b84-cfc043da0372 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:26.676726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:26.677717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:26.678900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:26.685070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 414/802] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:26.686834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:26.697789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:26.731455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:26.732278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.732565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.732915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] PATH_INFO: `/v3/services/network` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.733577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:26.734955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:26.740045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6336cedd-16af-4155-adb6-446719927b52 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:26.749646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:26.754682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:26.755538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 389/803] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:26 2022] GET /identity/v3/services/network => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:26.756329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe97cb9-7c9a-409f-8a4a-a6e974866b1c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.756329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe97cb9-7c9a-409f-8a4a-a6e974866b1c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.756329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe97cb9-7c9a-409f-8a4a-a6e974866b1c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.756329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 390/804] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:26.761504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.761504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.761504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.780720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:26.789082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:26.836093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc1a2c-7161-4b71-8a39-4d8ee737d1fa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:26.878080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:26.878764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:26.879053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:26.879329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:26.880178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:26.880492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:26.883023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:26.908708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 391/805] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:26.908708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:26.919240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 415/806] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Jul 18 10:23:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:26.926525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.032984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.036369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e557c-6f1b-4a09-983b-30d819784bde None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.040019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc07c5-2d9b-4c81-9328-a91f891c06d6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.052267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.074500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.075058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.075314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.075604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.075892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.076383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.076728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.076996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.077718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:27.078333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.079468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.084929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 392/807] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:27.085934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.089375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.091123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:27.110664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:27.111583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:27.112747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:27.112747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:27.112747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:27.112747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:27.112747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 416/808] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:27.113039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.221255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f7b1e-7fa6-46d8-8fad-ba06c1d32afc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.221463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c09a-302e-495e-aed3-e268e142fa42 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.261299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.262480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.262987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.263470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.265829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.266508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.267122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.267122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.268220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] RBAC: Adding query filter params (type=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:27.268544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.269997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.276878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 417/809] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:27.277983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.278627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:27.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:27.290669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:27.291605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:27.293017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.294738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 393/810] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:27.295667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.340720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.402899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68969ae-1805-4af8-ab42-c5c87388580c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.406260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29f930-a347-48bd-8280-42b69bc63e5f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.442060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.442733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.443024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.443306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.444037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:27.444407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.445379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.445830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.446483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.446984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.447457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.448404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.449999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.463143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 418/811] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:27.463143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.536878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.565869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 394/812] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:27.567703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.596963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aafc71-2dc3-4508-8f7e-1872e89ce20f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.685603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.686708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.688292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.688823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.689837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:27.690558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.693513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55845e74-86ff-4b15-be8a-8672f3fb68c6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.707064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 419/813] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:27.712724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b3c7-b5e2-4284-b927-8e79a032906e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.712724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b3c7-b5e2-4284-b927-8e79a032906e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.712724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b3c7-b5e2-4284-b927-8e79a032906e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.715741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 420/814] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:27.717297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.717595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.717881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.737431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:27.738757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:27.755721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.765819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b06b81-7278-4684-9c8f-9c80a046950a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.812606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.814553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.814850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:27.815419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:27.830355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 421/815] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:27.830355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.830610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:27.830610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:27.858149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:27.859218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:27.860088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 395/816] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:27.860560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:27.906100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:27.959938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4e727-6f49-491a-8817-1aa103f5f655 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.971009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80f102-1a06-43ac-b191-938822cc8cb5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:27.998175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:27.999314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:27.999832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.000312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.008941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.009744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.010125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.010413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.011106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.012372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.012485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.014226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:28.024364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:28.025462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:28.026301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 396/817] 10.222.0.47 () {56 vars in 1129 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:28.027126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:28.035566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:28.036649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:28.037795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:28.037795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:28.037795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:28.037795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:28.044475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 422/818] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:28.045515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.050018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.136850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b06fa-3863-4713-8636-f79d129717fb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.146684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbac983-ac6b-4255-8587-9ac690561fe4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.158011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.173875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.174738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.175189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.175527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.176323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:28.176640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.178094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.186001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.186676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.187014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.187300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.188017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:28.188333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.189730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.195974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 397/819] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:28.195974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.199859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 423/820] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:28.204504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.290251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.311856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806227-5f31-472c-8807-eea90b3b29ba None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.316608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151669b-7184-485f-8677-f445fdeac5ea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.350318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.351761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.352070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.352346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.353180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:28.353483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.354035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.354861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.354962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.355196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.355476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/5554e4999dc04da682c7f1cbd5f75abc/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.357602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=5554e4999dc04da682c7f1cbd5f75abc, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.358918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.367649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 398/821] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:28.368566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.442424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 424/822] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:28 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/5554e4999dc04da682c7f1cbd5f75abc/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:28.444046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.474986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569446-a570-478c-8424-d6fa894168c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.503304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.522177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.523767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.524294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.524867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.525867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:28.526415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.528119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.540014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 399/823] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:28.542762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.546272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.587903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a301f85-078c-48c7-8807-1f0dac23ff53 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.633714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.633714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.634019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.634224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.642975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.643595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:28.660246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:28.661324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:28.662250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 425/824] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:28.668236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.683131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab8b1-3f78-4e92-9e40-e1aa8cfca941 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.726249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18784-e240-40c6-9184-fbd216eb52cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.726808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18784-e240-40c6-9184-fbd216eb52cc None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.727088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18784-e240-40c6-9184-fbd216eb52cc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.727371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18784-e240-40c6-9184-fbd216eb52cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.728122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a18784-e240-40c6-9184-fbd216eb52cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.729516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a18784-e240-40c6-9184-fbd216eb52cc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.768869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.775537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab076b-76bf-47d4-803c-1ba1d0ab2b3b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.809317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:28.810084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.810359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.810635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.811465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:28.811814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:28.813256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a12b-a58a-4e0d-a446-1f6e0b289713 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:28.830601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 426/825] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:28.831860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91985ff-73c2-40fb-b75a-859726ce9a5a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.832165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91985ff-73c2-40fb-b75a-859726ce9a5a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.832692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91985ff-73c2-40fb-b75a-859726ce9a5a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.834102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 427/826] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:28.836408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91985ff-73c2-40fb-b75a-859726ce9a5a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:28.845759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 400/827] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:28.847543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:28.848045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:28.848543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:28.873959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:28.875794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:28.948092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:28.989428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91985ff-73c2-40fb-b75a-859726ce9a5a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:28.989845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 401/828] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:29.014989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.030209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.031032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.031411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.031791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.060803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.120271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.121763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.153981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340429fb-d676-4ac1-8e93-8041d654320e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.199209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.200826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.201329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.202132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.220638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:29.227488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 428/829] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:29.231024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:29.232199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:23:29.233088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:29.233740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.237369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 402/830] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:29.246380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.297420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.298995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.348366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9cbe8-52a4-4020-ab94-0d3f85b05bf9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.355046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d83bc-69c7-42ed-b6ff-64411234618d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.384374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.385244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.385484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.385777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.399164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.399164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.399164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.399164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.399560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:29.399560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.399560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.399738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.399936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.407299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 403/831] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:29.412671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:29.418190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:29.419136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:29.419998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:29.419998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:29.419998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:29.419998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:29.424025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 429/832] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:29.424946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.528211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6028b-511a-4cf0-a280-e6c4e6e89421 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.535749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.558271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c8155-7210-4280-a748-2a4e32da4451 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.561017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.563696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.564783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.565284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.565799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.598595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.599320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.599557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.599835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.600616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:29.600958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.602542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.620551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 430/833] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:29.621701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.705832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.707541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:29.727252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:29.728186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:29.729107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:29.729107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:29.729107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:29.729107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:29.729107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 404/834] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:29.734867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.749522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af1b5d-fe2d-42a2-ba9a-049fd42da77f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.793607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.794952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.795405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.796013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.807278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.809864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:29.827033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:29.828105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:29.829144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:29.835204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 431/835] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:29.836138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.851316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a7423-24c8-4053-b1c1-099fc64c758e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:29.867425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.886679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:29.887505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:29.887858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:29.888204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:29.889065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:29.889438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:29.890909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:29.908031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 405/836] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:29.909039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:29.929537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:29.970300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f278f-e8e7-466c-ba17-efd425abb4c3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.015540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.016350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.016761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.017043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.017834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:30.018166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:30.020566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:30.023027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae68d8-7539-4712-8ae5-0aef863cb886 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.041614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 432/837] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:30.043728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:30.064395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.065307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.065587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.065872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.075189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:30.076648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:30.093847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:30.094868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:30.095787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:30.099742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 406/838] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:23:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:30.100632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:30.159991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:30.179123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab2ddb-6cbd-4ac2-b96f-73cf7e7f0dc0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.190343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:30.213102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c41a33-92ee-4d8f-87fd-c662146fbb82 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.222194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.223219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.223526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.224089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.225260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aa1f5d0bba8a4178977c19b7503ef6a0) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:30.226288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:30.228418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:30.236387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 433/839] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/role_assignments?user.id=aa1f5d0bba8a4178977c19b7503ef6a0&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:30.267384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.268141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.268478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.268792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.269663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:30.270133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:30.271787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763775-06e2-44d8-9789-65578f1d6986 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:30.296109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 407/840] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:30.301884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:30.422126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1539ee-8a3c-4d1c-b955-d8e77260857e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.458824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.460456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.460992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.461471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.481017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:30.533678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:30.535473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:30.627163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e94a7c-a347-47c5-88ed-1e672da63091 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.627472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e94a7c-a347-47c5-88ed-1e672da63091 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.627642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e94a7c-a347-47c5-88ed-1e672da63091 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.630320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 408/841] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:30.635110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.635720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.636209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.650358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 434/842] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:23:30.654846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:30.656099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:30.657673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:30.709539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:30.736797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 409/843] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:30.753625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:30.769188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6dcd7-813d-4aac-b2bc-bd12f8a83a7f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.812477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.813702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.814192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.814662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.829248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:30.832198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:30.836005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:30.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:30.864069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:30.865021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:30.865021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:30.865021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:30.865021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:30.871404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 435/844] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:30.872514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:30.875249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251c6fb-c54b-4cdd-bf6e-e5d00aba0fb1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:30.912026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:30.913328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:30.913886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:30.914161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:30.925905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:30.934063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:30.935007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:30.935884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:30.940337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 410/845] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:30.941660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.001945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.006449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a81e9-6e2f-4236-8555-aec9a38b240a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.054062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76b2c7-deab-4383-9495-7d6973f780e7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.056358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.058318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.058929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.059596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.060602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.061139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.062859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1167b9-8fb5-4c55-925b-feaaf88735e9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.080842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 436/846] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.082524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b13ed-0541-4fa8-8836-bf12dbeb79d4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.082829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b13ed-0541-4fa8-8836-bf12dbeb79d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.083108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b13ed-0541-4fa8-8836-bf12dbeb79d4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.084432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 437/847] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:31.088602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.089530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b13ed-0541-4fa8-8836-bf12dbeb79d4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.097967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.098631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.099158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.103550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.103550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.103550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.103550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ec6f1-4302-4113-b994-34a4182d3dd6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.116721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 411/848] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.116721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.116721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.116721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.131597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:31.133524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:31.197665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b13ed-0541-4fa8-8836-bf12dbeb79d4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.203289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 412/849] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:31.204136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.241871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.242899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.243367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.243646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.250376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.251822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:31.266136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:31.267003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:31.267910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:31.272796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 438/850] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:31.274108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.298279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.319897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b516-2dec-4ffd-8ad2-63f0e600a5c4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.338672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.359732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.360839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.361135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.361416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.372762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.374207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.380988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0613-e95e-4e21-b3f3-ca044eec9ea9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:31.391780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:31.393017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:31.393986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:31.393986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:31.393986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:31.393986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:31.393986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 413/851] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:31.394254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.422727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.423538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.423827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.424104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.424790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.426252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:31.439080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:31.440153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:31.441023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 439/852] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:31.442937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.505717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9279-25f0-468d-a7b1-b2361041b496 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.535668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.541175 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.541836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.542120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.542392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.543236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.543547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.544987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.554903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df123d5f-a05d-44e9-ae7d-1b94e9d3a3e7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.558282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 414/853] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.559312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.592810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.602715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.603375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.603679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.604478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.604699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.605011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.606384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.618118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 440/854] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.618943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.679962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca580-0a5e-4f9c-8686-0458fa4908df None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.721616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.722608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.723091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.723562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.724486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.724912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a4135-c3f5-4991-8bea-1f7632b16b98 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.725665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.727399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.737148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 415/855] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.737992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.760829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.761438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.761709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.761992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.762818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=8721bfc5b53a421d99d1e75f84e639b3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.763132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.764716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.770940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 441/856] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/role_assignments?user.id=8721bfc5b53a421d99d1e75f84e639b3&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.771736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:31.785998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.858300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:31.867518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dd40c-7f1f-4ee7-8072-a1f58cce2b09 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.886286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6953c-fa67-495b-be07-3c7369b10061 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:31.908954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.910413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.910901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.911445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.921417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.923455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.924615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446798d-6652-4aba-939f-d6676786614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:31.925375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446798d-6652-4aba-939f-d6676786614e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:31.925660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446798d-6652-4aba-939f-d6676786614e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:31.925946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446798d-6652-4aba-939f-d6676786614e None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:31.926677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446798d-6652-4aba-939f-d6676786614e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:31.926989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446798d-6652-4aba-939f-d6676786614e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:31.930081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446798d-6652-4aba-939f-d6676786614e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:31.936783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 442/857] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:31.946525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:31.952848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:31.953766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 416/858] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:31.957520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446798d-6652-4aba-939f-d6676786614e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.052840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:32.092894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446798d-6652-4aba-939f-d6676786614e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:32.131429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:32.133148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.133647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.134157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.135220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:32.135746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:32.137416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bf308-2cba-4333-a5c2-f10a40d07747 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:32.156524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 443/859] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Jul 18 10:23:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:32.162521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.269964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e4f0-1ed7-4d47-90c2-f3ee45cf8e04 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:32.307161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:32.308108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.308648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.309127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.337581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:32.371216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a6b53-ed08-42f3-98ae-963396f3c6ce None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.371552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a6b53-ed08-42f3-98ae-963396f3c6ce None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.371785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a6b53-ed08-42f3-98ae-963396f3c6ce None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.375855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 444/860] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:32.381321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.382045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.382366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.389152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:32.390877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:32.399726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:32.400967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:32.485125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 445/861] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:32.495335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 417/862] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:23:32.500763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.502876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.573488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:32.575354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:32.618091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb65a03-68e4-4ac4-bcb2-036f66b9635b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:32.633769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b9f76-3148-4fb3-862e-ef44fc8cea02 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:32.655949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:32.657147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.657665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.658146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.670231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:32.672121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:32.680105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:32.680951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.681239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.681517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.682673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:32.684338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:32.692576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:32.693507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:32.694523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:32.694523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:32.694523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:32.694523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:32.700444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:32.701394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:32.702240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:32.703581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 446/863] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:32.704805 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 418/864] 10.222.0.47 () {56 vars in 1132 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:32.705880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.706090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.820808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590a16-1fcf-475a-8a0f-f6484d5e46f1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:32.837659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b1d0e-ed54-47f9-b7f6-131f1e0bdb92 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:32.839021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:32.856744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:32.869865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:32.870529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.870819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.871153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.871827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:32.872141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:32.873749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:32.879027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:32.879930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:32.880481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:32.881460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:32.882452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:32.883374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:32.885288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:32.894746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 447/866] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:32.894746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:32.895187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 419/866] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:32.896941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:33.007089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d759-73ff-4a47-860e-147091e7feb8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:33.030577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f6c2-ef92-4486-bf2a-c5030ae85a72 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:33.052654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:33.053533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:33.053823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:33.054145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:33.061782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:33.063787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:33.077474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:33.078570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:33.078948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:33.079305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:33.080091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:33.080470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:33.080957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:33.081845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:33.083003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:33.083445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:33.083515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 448/867] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:33.089179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:33.092774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 420/868] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:33.134953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:33.212239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d504f0-c965-46c6-bd54-6759b5a47e58 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:33.251113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:33.251113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:33.251113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:33.251113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:33.251946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:33.252261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:33.253751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032929-683e-41b7-b917-0246bcde36ab None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:33.272883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 449/869] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:33.274541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:33.331416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:33.387326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81365e0b-2a75-4690-a8e6-133a0d3ac5ce None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:33.422036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:33.422684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:33.422964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:33.423232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:33.424061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=5554e4999dc04da682c7f1cbd5f75abc) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:33.424368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:33.425796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0dbc5b-afcd-4443-83fc-c63dff3549c0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:33.432605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 421/870] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:33 2022] GET /identity/v3/role_assignments?user.id=5554e4999dc04da682c7f1cbd5f75abc&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:34.031909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88250a-12c1-461b-b6c8-3a10096696f9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.032530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88250a-12c1-461b-b6c8-3a10096696f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.032530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88250a-12c1-461b-b6c8-3a10096696f9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.034123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 450/871] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:34.039722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.040143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.040319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.057370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:34.058542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:34.099553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ca7d4-7400-4868-a25c-b3cf580633aa None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.099887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ca7d4-7400-4868-a25c-b3cf580633aa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.100117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ca7d4-7400-4868-a25c-b3cf580633aa None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.102292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 451/872] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:34.107382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.107629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.107913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.128188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:34.129434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:34.140990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 422/873] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:34.159622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:34.204917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:34.216710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 452/874] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:34.235832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:34.260529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:34.283318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d7d24-f7c3-4c97-94d1-a12631a34973 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:34.332893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:34.335112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.336070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.336577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.346867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1e83b-12fe-469f-a2dd-233791dc500e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:34.351970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:34.362280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:34.363289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:34.364199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:34.368788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 423/875] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Jul 18 10:23:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:34.372581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:34.390685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396b21-3578-440a-9202-095421d99228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:34.391415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396b21-3578-440a-9202-095421d99228 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.391638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396b21-3578-440a-9202-095421d99228 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.391924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396b21-3578-440a-9202-095421d99228 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.392656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77396b21-3578-440a-9202-095421d99228 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:34.394056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77396b21-3578-440a-9202-095421d99228 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:34.468726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 453/876] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:34 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:34.493428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:34.497607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad3a59-64f4-4322-9dee-b2c1b74fa6dc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:34.545292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:34.545883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.546225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.546500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.547269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:34.547579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:34.549140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:34.560446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 424/877] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:23:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:34.566879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396b21-3578-440a-9202-095421d99228 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:34.663443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:34.682000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396b21-3578-440a-9202-095421d99228 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:34.724980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:34.726192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.726701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.727200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.738080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:34.739887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:34.759095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:34.760036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:34.760941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:34.760941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:34.760941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:34.760941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:34.763160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 454/878] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:34.768990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:34.875509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb6510-d044-43d4-8fb6-c33bafeecff9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:34.910849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:34.911513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:34.911798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:34.912135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:34.912934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:34.913243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:34.914602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:34.926598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 425/879] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:34.933019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.044422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af898ad-6c4b-490d-ac3c-4aac5631d254 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:35.081284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:35.082095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.082629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.082902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.084879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.090787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:35.092251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c06efbfd-b8b0-451b-8ba3-7003171f1dac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:35.108076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:35.109100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:35.110041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 455/880] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:23:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:35.117027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.141430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.228048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290327e-a023-4fdc-933d-05e9031d47d1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:35.249309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55741a7-e6c7-49f5-b97e-e0a5274c000d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.249652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55741a7-e6c7-49f5-b97e-e0a5274c000d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.249903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55741a7-e6c7-49f5-b97e-e0a5274c000d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.255302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 456/881] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:35.257662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.258029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.258390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.279535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:35.280208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.280498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.280808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.281662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:35.281974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:35.283399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:35.285824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:35.288669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:35.307857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 426/882] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Jul 18 10:23:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:35.313788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.362488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 457/883] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:35.386323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.402547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.432845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf53d8-8ab0-4329-b4d0-e160ca4e3579 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:35.469408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:35.470085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.470367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.470645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.494092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946c3a-a3c1-4238-b1d8-dc34363b2564 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:35.518853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.531618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c3b4e-4ab2-40ea-af98-90dff49c7bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:35.533058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c3b4e-4ab2-40ea-af98-90dff49c7bb2 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.533346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c3b4e-4ab2-40ea-af98-90dff49c7bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.533621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c3b4e-4ab2-40ea-af98-90dff49c7bb2 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.534349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c3b4e-4ab2-40ea-af98-90dff49c7bb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:35.535740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c3b4e-4ab2-40ea-af98-90dff49c7bb2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:35.547118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:35.548861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:35.633168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 458/884] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:35 2022] POST /identity/v3/services => generated 269 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:35.635269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff873-2686-4759-9899-b99b31534f9e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.635579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff873-2686-4759-9899-b99b31534f9e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.635951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff873-2686-4759-9899-b99b31534f9e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.637876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.638434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 459/885] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:35.644216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.644216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.644551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.672501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:35.674196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:35.681239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 427/886] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:35 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:23:35.686701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.784259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 460/887] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:35.806185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.821502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d0737-b695-4a14-a2cd-9b4ca299a295 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:35.872140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:35.872945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.873234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.873513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.884337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:35.885826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:35.904584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:35.905560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:35.906421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:35.906421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:35.906421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:35.906421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:35.906421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 428/888] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:35.914196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:35.936012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3be062-2e67-4c7a-b30d-a9a12ef27af5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:35.944766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.951095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:35.979225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:35.980341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:35.980862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:35.981363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:35.982230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:35.983808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:35.996764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:35.997822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:35.998656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:36.001519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 461/889] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:36.012209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:36.035865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353ddd9-ae8a-4039-ada3-8572a294373b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.082063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.082985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.083514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.083790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.085773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:36.086079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.087740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.103436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 429/890] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:36.108996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:36.122391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24334464-bec9-4edc-8c88-45b38bb07177 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.164594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.165314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.165604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.165887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.166603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:36.166919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.168546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.178613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 462/891] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:36.179327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:36.225272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:36.230156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:36.246142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102e200-2cbf-438f-87f7-b13e5e64a2d8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.298266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb608fbd-c711-49cb-8319-7e03cb9288f5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.303317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.304554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.305004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.305425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.315309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.317598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.336866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.337740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.338146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.338525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:36.339110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:36.340136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:36.341291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:36.341834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:36.342139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.343644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 430/892] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:36.343771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.352735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:36.360786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 463/893] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:36.473962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060796b-b0b7-4cc1-93d4-cbfa665c8c1a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.513593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.514261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.514547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.514825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.515618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:36.515932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.517375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ab3cd-bcce-47e2-b4e1-14925e50014f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.529889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 431/894] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:36.535826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:36.646926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3722b0-57dd-4a12-a373-f4f8b3d9331b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.649398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:36.684147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.684838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.685123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.685337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1d030-0ff3-4da9-b83e-f900e84f433b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.685442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.685649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1d030-0ff3-4da9-b83e-f900e84f433b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.685922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1d030-0ff3-4da9-b83e-f900e84f433b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.686470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] RBAC: Adding query filter params (role.id=ca9ad274f82348179efed0c66e246b84, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=8721bfc5b53a421d99d1e75f84e639b3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:36.686958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.688446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.688553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 432/895] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:36.692425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.692763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.693052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.696696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 464/896] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/role_assignments?user.id=8721bfc5b53a421d99d1e75f84e639b3&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ca9ad274f82348179efed0c66e246b84 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:36.712132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:36.713409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:36.715742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:36.792940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 433/897] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:36.816248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:36.926093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d81a7-acb3-47d4-b1c8-d534513687e9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:36.962961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:36.963825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:36.964116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:36.964400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:36.965100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:36.967740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:36.980727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:36.981757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:36.982574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:36.987154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 465/898] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:36.989789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:37.096265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7eba8-50b5-42b1-bc0d-30c766f27fe1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:37.132046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:37.132741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:37.133034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:37.133311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:37.134029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:37.134336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:37.135704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:37.140905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 434/899] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:37.150924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:37.183547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:37.257139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe4c1-0813-4da1-9062-a70338b0e4c1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:37.295332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:37.296001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:37.296290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:37.296573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:37.297325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:37.297640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:37.299434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:37.308794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 466/900] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:37.311283 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:37.325020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:37.422997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f1c6-0313-4e1f-b4d1-4c6dd12d9a6d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:37.457886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:37.458546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:37.458830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:37.459110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:37.460747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] RBAC: Adding query filter params (interface=public, service_id=1ef769490ba04a32b185bdcdd0d0f808, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:37.460974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:37.462403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c585023-e4aa-4fdb-a16c-d01a8514ee69 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:37.468163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 435/901] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:37 2022] GET /identity/v3/endpoints?service_id=1ef769490ba04a32b185bdcdd0d0f808&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:37.473467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:37.574303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:37.581716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8f9a-f243-4380-89d8-e3906c5770e1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:37.621568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d936e6-4635-4b93-a50a-e90774f49b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:37.622242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d936e6-4635-4b93-a50a-e90774f49b48 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:37.622531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d936e6-4635-4b93-a50a-e90774f49b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:37.622815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d936e6-4635-4b93-a50a-e90774f49b48 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:37.623499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d936e6-4635-4b93-a50a-e90774f49b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:37.624888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d936e6-4635-4b93-a50a-e90774f49b48 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:37.631808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 467/902] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:37 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:23:37.918763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15848b-a8bd-4bd2-bd0c-dc7c6e0fbeae None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:37.919473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15848b-a8bd-4bd2-bd0c-dc7c6e0fbeae None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:37.919970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15848b-a8bd-4bd2-bd0c-dc7c6e0fbeae None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:37.921845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 436/903] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:37.927237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:37.927537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:37.927819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:37.946778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:37.947985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:38.020329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 468/904] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:38.044650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15848b-a8bd-4bd2-bd0c-dc7c6e0fbeae None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:38.120373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:38.155777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15848b-a8bd-4bd2-bd0c-dc7c6e0fbeae None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:38.191618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:38.192355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.192640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.192948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.193608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:38.195527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:38.208280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:38.209234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:38.210692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 437/905] 10.222.0.47 () {56 vars in 1132 bytes} [Mon Jul 18 10:23:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:38.218115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:38.311929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:38.325565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e209b-3410-40db-9fd5-3c5748466fbe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:38.361268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1876081-4475-413e-9ae6-077252547dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:38.361942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1876081-4475-413e-9ae6-077252547dae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.362229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1876081-4475-413e-9ae6-077252547dae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.362510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1876081-4475-413e-9ae6-077252547dae None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.363232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1876081-4475-413e-9ae6-077252547dae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:38.363548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1876081-4475-413e-9ae6-077252547dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:38.364951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1876081-4475-413e-9ae6-077252547dae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:38.371314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 469/906] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:38.378028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:38.483642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e71de4-4c40-4482-98f5-93b18fbe2ff0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:38.515068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:38.518424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:38.519088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.519372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.519654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.520367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:38.520694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:38.522582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4960-89ca-4448-99f0-0a994b071bdf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:38.527545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 438/907] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:38.533250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1876081-4475-413e-9ae6-077252547dae None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:38.554635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f498e92-d68f-4836-afac-a406f9d30a34 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.554923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f498e92-d68f-4836-afac-a406f9d30a34 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.555210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f498e92-d68f-4836-afac-a406f9d30a34 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.556741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 439/908] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:38.564151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.564877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.565488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.586095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:38.587294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:38.641247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1876081-4475-413e-9ae6-077252547dae None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:38.658722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 440/909] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:38.682809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:38.686360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:38.687049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.687334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.687618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.688417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] RBAC: Adding query filter params (interface=public, service_id=e440a481aba9493e88dfb30584a9b224, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:38.688782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:38.690179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:38.698414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 470/910] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:38 2022] GET /identity/v3/endpoints?service_id=e440a481aba9493e88dfb30584a9b224&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:38.703960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:38.724445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:38.814789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfbd00-43b8-43af-bd39-7626b548396f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:38.828486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b05f-5ad5-4d23-b062-60376b8f8831 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:38.853287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:38.862461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:38.863126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.863415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.863694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.864717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:38.866064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:38.872902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50611a9-eecc-418f-8718-c8f666b5e3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:38.873571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50611a9-eecc-418f-8718-c8f666b5e3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.873859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50611a9-eecc-418f-8718-c8f666b5e3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.874141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50611a9-eecc-418f-8718-c8f666b5e3a3 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.874832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50611a9-eecc-418f-8718-c8f666b5e3a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:38.876203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50611a9-eecc-418f-8718-c8f666b5e3a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:38.882443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 471/911] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:38 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:23:38.883730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b8942-011b-4117-90d2-9ae3c5386016 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.884030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b8942-011b-4117-90d2-9ae3c5386016 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.884315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b8942-011b-4117-90d2-9ae3c5386016 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.885970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 472/912] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:38.894746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:38.895066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:38.895322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:38.915273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:38.916517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:38.964026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 441/913] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:38 2022] POST /identity/v3/services => generated 241 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:39.020852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 473/914] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:39.041625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:39.163679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bad57-b33b-4226-8e20-96256cbdaeb6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:39.199771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:39.200947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.201402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.201863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.213858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:39.222296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:39.223212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:39.224051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:39.226273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 442/915] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:39.232287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:39.241402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:39.353242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0f147-6038-4953-9652-444df82f1d28 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:39.392121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:39.396711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.396711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.396711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.396711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:39.396711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:39.396711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:39.408733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 474/916] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:39.417851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:39.466933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:39.523899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20830f88-4115-4ad8-a649-d7d0de977c88 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:39.558357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:39.559604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.560096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.560620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.572291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:39.574055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8809ec96-5b2f-4788-82c9-80a2082b13d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:39.592868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:39.593795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:39.594688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:39.594688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:39.594688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:39.594688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:39.597040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 443/917] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:39.603706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:39.678070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:39.714026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c89ec-eb22-40dc-a84c-4b3dff6b935f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:39.749091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:39.749763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.750054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.750338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.751074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:39.751388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:39.752787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:39.768794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 475/918] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:39.770724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fee741-624f-4b50-b238-7bb25119eefa None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.771013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fee741-624f-4b50-b238-7bb25119eefa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.771188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:39.771280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fee741-624f-4b50-b238-7bb25119eefa None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.773879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 444/919] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:39.778349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05962327-d813-456c-82c8-86ae78de6644 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.778640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05962327-d813-456c-82c8-86ae78de6644 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.778921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05962327-d813-456c-82c8-86ae78de6644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.795914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-05962327-d813-456c-82c8-86ae78de6644 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:39.797211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-05962327-d813-456c-82c8-86ae78de6644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:39.877477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 445/920] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:39.893807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cc71b-c21b-4d06-b917-001cbe5a6f5f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:39.894155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05962327-d813-456c-82c8-86ae78de6644 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:39.929211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:39.929659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:39.939156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:39.940137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:39.944161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:39.944161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:39.951409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:39.953636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:39.973219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:39.974232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:39.975785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:39.981096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 476/921] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:39.996092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.005533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05962327-d813-456c-82c8-86ae78de6644 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.056125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.057757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.058745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.059550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.060472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.063170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:40.081869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:40.083233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:23:40.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:40.088731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 446/922] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:40.091904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.117162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a04f4e-878b-4553-afdb-4b513d0d0adf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.153954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.154633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.154920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.155202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.156111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:40.156429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.157914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.179096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 477/923] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Jul 18 10:23:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:40.184344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.217775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:40.229392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa81ea-e903-44b8-9376-dba87ce4fac1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.243134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:40.283298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.283977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.284265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.284544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.285388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:40.285705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.288082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.299827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 447/924] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:40.304579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.313489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016abe-6c09-40cc-86a8-d15667a3532c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.352903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.354214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.354747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.355244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.366536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.368228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:40.387348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:40.388298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:40.389228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:40.389228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:40.389228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:40.389228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:40.391372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 478/925] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:40.399004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.426505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4b921-12e7-4e72-af6e-e182e446a144 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.464654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.465760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.466275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.466749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.467757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:40.468263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.469890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.480777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 448/926] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Jul 18 10:23:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:40.482420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.504218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:40.524908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:40.528985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5f17e-9b62-4208-b436-e24d706121f5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.575984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.577243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.577917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.579040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.579784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:40.580101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.581864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.596436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 479/927] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:40.602479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.608203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef32c-29b8-4e34-b5a2-1554cedd7e9e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.653693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.654793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.655319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.655814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.657332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.659409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.721846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37870ca-74f6-4176-b9bb-3830d60d43d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.757523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.762493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.762493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.762493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.762759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 449/928] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:40.777334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.779262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:40.796311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:40.797846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:40.798996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:40.805710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 480/929] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:40.807413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:40.895298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:40.930419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4571-ee26-4a08-a70c-39ac82baeed2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:40.970196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:40.971144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:40.971771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:40.972320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:40.973318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:40.974104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:40.976253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b6773-ab27-471f-be8f-bca4f9476332 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:40.989923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 450/930] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:40.995865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.007450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26b837-2c4e-4710-aab6-6079d02bffdd None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.007739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26b837-2c4e-4710-aab6-6079d02bffdd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.008011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26b837-2c4e-4710-aab6-6079d02bffdd None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.009519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 451/931] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:41.015460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.015735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.016008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.018784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.035924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:41.037168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:41.122699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5f8c-2421-4666-a042-26ef44b29a2f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.128766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 452/932] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:41.149636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.162689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588f4fc-4844-4e75-9c5f-41ad3b7586c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.164000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588f4fc-4844-4e75-9c5f-41ad3b7586c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.164284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588f4fc-4844-4e75-9c5f-41ad3b7586c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.164562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588f4fc-4844-4e75-9c5f-41ad3b7586c5 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/8721bfc5b53a421d99d1e75f84e639b3/roles/ca9ad274f82348179efed0c66e246b84` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.167601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588f4fc-4844-4e75-9c5f-41ad3b7586c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=8721bfc5b53a421d99d1e75f84e639b3, role_id=ca9ad274f82348179efed0c66e246b84)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.169353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588f4fc-4844-4e75-9c5f-41ad3b7586c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.213884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.253426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ca4f9-e277-4c19-8f40-576e23c7106a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.272972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 481/933] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:23:40 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/8721bfc5b53a421d99d1e75f84e639b3/roles/ca9ad274f82348179efed0c66e246b84 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:23:41.274576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd257a5-faf6-43f7-ab8e-469986a7fbec None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.274867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd257a5-faf6-43f7-ab8e-469986a7fbec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.275147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd257a5-faf6-43f7-ab8e-469986a7fbec None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.280891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 482/934] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:41.282806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.283104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.283385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.303690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.304468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.304774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.305052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.305741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.307553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.315186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:41.316867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:41.322779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:41.323786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:23:41.324658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:41.329638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 453/935] 10.222.0.47 () {56 vars in 1132 bytes} [Mon Jul 18 10:23:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:41.330701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.425240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 483/936] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:41.451505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.453610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d42948-a3a4-4692-ab8c-ab6d4d9c9994 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.464031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.485119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.493320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.494000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.494290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.494569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.495286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:41.495595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.497009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.507076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 454/937] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:41.511872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.568185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fff937-e18d-4ebd-9f3e-3a5e90897c2a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.605547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.606369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.606654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.606931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.607595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.608984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:41.622102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:41.623126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:41.623948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 484/938] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:41.630437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.637435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb831-cb6a-4373-b23c-229d96ee9f0d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.679741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.680408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.680718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.680997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.681782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:41.682243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.683622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.695170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 455/939] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:41.696602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.726597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.758323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f43435-cb00-49e1-a741-105778a8a1a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.770277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.802405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.804152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.805565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.806025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.807715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:41.808325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.810270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.822303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 485/940] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:41.824362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:41.834820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cc02-ccfd-4fc7-a874-250e4102bbdf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.873489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.874314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.874709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.875077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:41.875853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:41.877373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:41.949620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ff2bc-7b1f-498c-9296-bed0da0dc570 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:41.993244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 456/941] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:41.994823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:41.997745 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:41.998726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:41.999239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:41.999689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:42.000599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:42.001179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:42.002791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:42.010297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 486/942] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:42.023002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:42.149167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3788b-e259-4d86-b292-a669f72f784e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:42.157654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:42.183102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:42.183774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:42.184062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:42.184346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:42.185173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] RBAC: Adding query filter params (interface=public, service_id=d2a34441291345d49c888e6a22f2fded, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:42.185494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:42.186860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717d28f-6ac5-46d1-aae1-a756077e7088 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:42.192344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 457/943] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:42 2022] GET /identity/v3/endpoints?service_id=d2a34441291345d49c888e6a22f2fded&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:42.198447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:42.306116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ed6b-6525-4895-863a-b8969bfe217c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:42.343321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c9408-62ac-4e09-9c58-2b91ba1bbc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:42.344283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c9408-62ac-4e09-9c58-2b91ba1bbc68 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:42.344783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c9408-62ac-4e09-9c58-2b91ba1bbc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:42.345270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c9408-62ac-4e09-9c58-2b91ba1bbc68 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:42.346137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c9408-62ac-4e09-9c58-2b91ba1bbc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:42.347692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c9408-62ac-4e09-9c58-2b91ba1bbc68 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:42.354897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 487/944] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:42 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:23:42.964160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003a03-a628-4694-a9d5-fe2bdd0fe259 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:42.964734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003a03-a628-4694-a9d5-fe2bdd0fe259 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:42.964734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003a03-a628-4694-a9d5-fe2bdd0fe259 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:42.966537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 458/945] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:42.971740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:42.972045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:42.972334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:42.989864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:42.991102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:43.050169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:43.062375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 488/946] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:43.085883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003a03-a628-4694-a9d5-fe2bdd0fe259 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:43.194829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003a03-a628-4694-a9d5-fe2bdd0fe259 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:43.236614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:43.237240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.237529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.237809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.238531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:43.239975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:43.242028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING keystone.server.flask.common [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 10:23:43.307788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:23:43.307788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 10:23:43.307788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [parameters: {'id': '0211b588a4914d77a69a3be5747af431', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 10:23:43.307788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5424bea4-420d-4241-8903-17de2b78a29d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0211b588a4914d77a69a3be5747af431.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0211b588a4914d77a69a3be5747af431. Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:23:43.315353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 10:23:43.320090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() Jul 18 10:23:43.321099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:23:43.321990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:23:43.322872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:23:43.323694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'id': '0211b588a4914d77a69a3be5747af431', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:43.324549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:43.327369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0211b588a4914d77a69a3be5747af431. Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.329533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 459/947] 10.222.0.47 () {60 vars in 1140 bytes} [Mon Jul 18 10:23:43 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:43.330845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:43.359332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b521d-535d-44ed-ab5f-a6843cf128a1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.359648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b521d-535d-44ed-ab5f-a6843cf128a1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.359996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b521d-535d-44ed-ab5f-a6843cf128a1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.368974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 460/948] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:43.370343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.370631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.370906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.391093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:43.393219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:43.420547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:43.432007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb2b64-8e93-4bc3-8dac-06fcd10d165e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:43.470747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:43.471543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.471830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.472103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.480880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 461/949] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:43.486391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:43.488037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:43.500223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:43.502505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:43.503393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:43.504264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 489/950] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:43.511296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:43.564233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:43.622514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66ec1-506c-426b-a84c-8b5ccbbfcede None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:43.625886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5db5b-2c87-4af0-908d-5e3fb758b605 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:43.663828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:43.664547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.664870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.665153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.665908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:43.666272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:43.667688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:43.669702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:43.670877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.671948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.672431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.684947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:43.693181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 490/951] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:43.694750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:43.695732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:43.696636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:43.699758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 462/952] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Jul 18 10:23:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:43.706240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:43.828272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4bd1c-4d6a-4bdc-bcf0-9facb9728c8d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:43.841818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:43.867287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:43.868901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:43.869398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:43.869901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:43.870895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:43.872537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:43.874180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:43.885714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 491/953] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:23:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:43.891777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:43.914649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.001777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d1571-9583-4a49-a3b0-ec20f566bcae None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.036595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.037463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.037812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.038064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.049096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.050672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a875c6da-e0d5-4afb-a8d1-3f97237a15d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:44.070173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:44.071138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:44.072036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:44.072036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:44.072036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:44.072036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:44.072036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 463/954] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:44.078667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.188563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c9e7-4fd4-4da6-8d1a-30fc7f6fa194 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.199675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530eab-fcd3-4bb9-9571-73220a293872 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.199970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530eab-fcd3-4bb9-9571-73220a293872 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.200256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530eab-fcd3-4bb9-9571-73220a293872 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.201924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 464/955] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:44.206881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.207174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.207710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.228669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:44.229912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:44.230542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.231435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.232189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.233730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.234685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:44.235201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.236802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.254079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 492/956] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:44.259922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.300916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.311192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.317863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 465/957] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:44.343520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.398918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcdd38-bad6-42e3-8641-3011fd2d9e64 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.436968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.438303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.438820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.439354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.449990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.451870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.469832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ddb4-d60a-441a-b050-a92c9b67feaf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3d0ba67e-84f7-489f-a5f8-c3dad9d53fbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:44.470142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:44.471053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:23:44.471985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:44.475014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 493/958] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:44.476662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d02f4-7e9f-4dae-ac0a-a0930f8b42fb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.476985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d02f4-7e9f-4dae-ac0a-a0930f8b42fb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.477270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d02f4-7e9f-4dae-ac0a-a0930f8b42fb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.478744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 494/959] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:44.482481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d02f4-7e9f-4dae-ac0a-a0930f8b42fb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.511586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.512376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.512663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.512955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.513612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.514972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e5f777af-8982-4364-b875-5255553bd54d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:44.527712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:44.533927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:44.534793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 466/960] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:44.535603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.535603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.535603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.548167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:44.549380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:44.592268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.606660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d02f4-7e9f-4dae-ac0a-a0930f8b42fb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.613511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.623718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 467/961] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:44.624535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.646106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.647952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.648469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.648868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.649783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:44.650183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.651686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.674973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 495/962] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:44.674973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.735910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d866-79f6-4d02-b65b-1b911edc159e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.773307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.774193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.774481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.774757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.775469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:44.775779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.777181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.781349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351bf8b-b05d-4dba-9553-7c16f4e022b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.782209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 468/963] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:44.782802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.821745 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.822687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.823067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.823433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.824181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.825667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:44.837997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:44.839198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:44.840228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:44.841555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 496/964] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:44.842079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:44.852145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.864033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:44.895259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d902816-0999-4aff-90ed-7b64a1460186 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.933715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.934411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.934708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.934988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.944864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee61bc-c2bb-40ca-bb87-7a428b107f12 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:44.985595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:44.986724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:44.987081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:44.987418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:44.988198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:44.988582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:44.990051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:44.995091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 497/965] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:44.995972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.019185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.021419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.087535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:45.093537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:45.111458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f12b-5cc9-47d5-8949-de0457a25639 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.143051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 469/966] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:23:45.149692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.162524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.163339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.163636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.163918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.164641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:45.165134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.166516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.172554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 498/967] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:45.178446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.271238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecde08-7306-45a0-8514-cea1bb84b922 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.297738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a16f88-57b0-4237-a20d-f362e1a6828b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.322916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.324655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.325492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.325980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.336970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.337648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.339418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.339418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.339418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:45.339418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.340821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.341226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.343593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.351220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 499/968] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:45.352284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.361614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:45.367735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:45.377967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:45.378893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:45.378893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:45.378893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:45.378893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:45.378893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 470/969] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:45.378893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.390916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:45.468315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccc415-e172-4167-bfe4-a4f0f30b4dfb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.489566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9109ba9-ff5d-4181-8ebf-b5e6bdef0f4a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.509642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.510482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.510667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.510951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.511696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.513130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.534992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.535736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.536021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.539746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.539746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:45.539746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.539746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.551299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 471/970] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:45.557824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.607959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 500/971] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:45.640915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:45.674449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5681f1-8329-4728-a337-17307aa9f374 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.728449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.729642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.730189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.730667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.738961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.741359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0a250fa8-1034-41ff-9b76-5f0602d358d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:45.758376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:45.759412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:45.760325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:45.763025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 472/972] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:45.768805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.838461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:45.879893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d3fea-2ac8-40ee-baf8-8efcf456a41d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:45.910914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338f2bb-e9db-418d-9d26-a6cefb9ed3c4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.911558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338f2bb-e9db-418d-9d26-a6cefb9ed3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.912160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338f2bb-e9db-418d-9d26-a6cefb9ed3c4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.915733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 473/973] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:45.920746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:45.920746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.920746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.920746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.921087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:45.921087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:45.921087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:45.921267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:45.921267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:45.921267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:45.933546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 501/974] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:45.939402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:45.942084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:45.943379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:46.032735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 474/975] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:46.055378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43b26-644d-432e-b307-ce4de123bbfa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:46.062366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:46.083970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:46.106377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:46.107070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:46.107357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:46.107629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:46.109039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] RBAC: Adding query filter params (role.id=ca9ad274f82348179efed0c66e246b84, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=8721bfc5b53a421d99d1e75f84e639b3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:46.109371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:46.111052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:46.124856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 502/976] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:45 2022] GET /identity/v3/role_assignments?user.id=8721bfc5b53a421d99d1e75f84e639b3&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ca9ad274f82348179efed0c66e246b84 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:46.138365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:46.184174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a41226-1f75-4101-8f66-5b3726d44dde None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:46.223210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:46.223899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:46.224183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:46.224459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:46.225793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:46.227252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:46.309580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:23:46.309580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 10:23:46.309580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [parameters: {'user_id': '07a56ca0320b49c89a06875011941764', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 10:23:46.309580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jul 18 10:23:46.317103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:23:46.323799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:23:46.324890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:23:46.325853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:23:46.326737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:23:46.327576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'user_id': '07a56ca0320b49c89a06875011941764', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:46.328402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 10:23:46.331295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:46.333895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 475/977] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Jul 18 10:23:46 2022] POST /identity/v3/users => generated 160 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:46.336277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:46.442826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72425bec-a343-4d05-a2f0-1b78db975d46 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:46.478959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:46.479818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:46.480109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:46.480387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:46.488262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:46.489750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cfaa51af-96e6-4909-8603-8686f8a31ebf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:46.504241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:46.505241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:46.506300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 503/978] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:46.511929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:46.620621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54bb8c-1f2c-4e87-b978-dfb882319f81 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:46.640513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:46.658704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:46.659736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:46.660353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:46.660866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:46.661882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:46.662586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:46.664564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1a4d9-2db2-4509-8297-7bb2d74a9375 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:46.705462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 476/979] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Jul 18 10:23:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:47.098478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096b7b4-72cc-461a-b97c-f8edb2cfa364 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:47.098803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096b7b4-72cc-461a-b97c-f8edb2cfa364 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:47.099154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096b7b4-72cc-461a-b97c-f8edb2cfa364 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:47.101528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 504/980] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:47.106666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef8368-b0fa-422f-961f-dd6f0f63fd5b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:47.107189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef8368-b0fa-422f-961f-dd6f0f63fd5b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:47.107375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef8368-b0fa-422f-961f-dd6f0f63fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:47.123893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f8ef8368-b0fa-422f-961f-dd6f0f63fd5b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:47.125421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8ef8368-b0fa-422f-961f-dd6f0f63fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:47.190395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 477/981] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:47.213165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096b7b4-72cc-461a-b97c-f8edb2cfa364 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:47.270727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:47.325724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096b7b4-72cc-461a-b97c-f8edb2cfa364 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:47.360138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7e1aa-ffee-4b4e-a1cd-d836cf8d563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:47.360758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7e1aa-ffee-4b4e-a1cd-d836cf8d563d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:47.361046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7e1aa-ffee-4b4e-a1cd-d836cf8d563d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:47.361325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7e1aa-ffee-4b4e-a1cd-d836cf8d563d None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:47.362063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7e1aa-ffee-4b4e-a1cd-d836cf8d563d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:47.363433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7e1aa-ffee-4b4e-a1cd-d836cf8d563d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:47.480604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 505/982] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:47 2022] POST /identity/v3/services => generated 242 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:47.757154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6bfe0-9d8f-4ee3-ba1a-5782652964df None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:47.757498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6bfe0-9d8f-4ee3-ba1a-5782652964df None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:47.757674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6bfe0-9d8f-4ee3-ba1a-5782652964df None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:47.759064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 478/983] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:47.764205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:47.764503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:47.764822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:47.785035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:47.786237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:47.863342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 506/984] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:47.885655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6bfe0-9d8f-4ee3-ba1a-5782652964df None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:47.993880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6bfe0-9d8f-4ee3-ba1a-5782652964df None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.001596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:48.029048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.029855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.030146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.030422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.031080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.032442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:48.045693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:48.046683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:48.047521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:48.054965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 479/985] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:48.055114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.086280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:48.156920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca048c-ef51-41c5-aab6-006a86529f09 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.191946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.192612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.192922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.193201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.194456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:48.194767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.196146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f6a7-d6ba-41fd-a0f6-317c6377788f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.205845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 507/986] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:48.206932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.312215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40285f-7849-4691-b420-f9c21abf7fe7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.312531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40285f-7849-4691-b420-f9c21abf7fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.312746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40285f-7849-4691-b420-f9c21abf7fe7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.314274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 508/987] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:48.319387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.319685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.319978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.321996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653a28b-800f-47f4-9819-90a5115cda70 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.336935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:48.338139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:48.359196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.360158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.360586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.361124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.361998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:48.362445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.364179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.370291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 480/988] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:48.375701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.411040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 509/989] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:48.416841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:48.432960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.460267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:48.487541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83168b48-df6a-4dbc-bfef-3377ede5f619 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.523117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.524225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.524673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.524981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.525781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] RBAC: Adding query filter params (interface=admin, service_id=d2a34441291345d49c888e6a22f2fded, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:48.526102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.527463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.532658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 481/990] 10.222.0.47 () {56 vars in 1249 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/endpoints?service_id=d2a34441291345d49c888e6a22f2fded&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:48.535803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec088fbe-7f39-4439-8aaf-c18b0ea2a489 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.538337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.575681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.576398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.576707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.577000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.577654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.579021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:48.591734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:48.592769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:48.593799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 510/991] 10.222.0.47 () {56 vars in 1105 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:23:48.599052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.656792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:48.662140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ce68b-8b06-451c-a977-41cd3d2acedb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.695393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:48.704038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451a5a6-5eb0-4c62-ab50-183b38ad9e38 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.705177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.706009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.707071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.707612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.708304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.709896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.721038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 482/992] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:23:48.741634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.742345 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.742629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.742905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.743631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:48.743943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.745424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160e27-15cf-41bc-88e7-57a2cb4a4e6a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.751295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 511/993] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:48.758203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:48.876065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfc283-a35c-4d11-9a5a-5c753ee15bbe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:48.909955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:48.910612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:48.910893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:48.911166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:48.911865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] RBAC: Adding query filter params (type=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:48.912714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:48.914114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373172-d396-4b60-b857-4007d6cbf9ff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:48.918739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 483/994] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:49.542769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65d5e8-3e26-464e-8dca-fc14e6a0a1d2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:49.543286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65d5e8-3e26-464e-8dca-fc14e6a0a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:49.543286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65d5e8-3e26-464e-8dca-fc14e6a0a1d2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:49.544655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 512/995] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:49.549685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:49.550075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:49.550246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:49.567276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:49.569037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:49.590093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:49.637015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 484/996] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:49.659709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65d5e8-3e26-464e-8dca-fc14e6a0a1d2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:49.763492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65d5e8-3e26-464e-8dca-fc14e6a0a1d2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:49.797411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:49.798117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:49.798406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:49.798688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:49.799340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:49.800732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:49.815171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:49.816147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:49.817415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 513/997] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:49.822840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:49.928832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d8a8c-1547-4e6f-b990-2c464a8fa031 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:49.964045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:49.964733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:49.965026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:49.965296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:49.966011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:49.966318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:49.967676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:49.977821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 485/998] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:49.979463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:49.999271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:50.083410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0824-646b-48a3-829e-36db6bbac1f9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:50.117758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:50.118416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.118695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.118965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.119689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:50.119997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:50.121407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:50.126425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 514/999] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:50.132213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:50.154539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:50.233156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f41d7-8bcf-4ea4-8c4f-4cd155f654a6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:50.271275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:50.272645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.272960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.273237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.274053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] RBAC: Adding query filter params (interface=public, service_id=0f304ae3aa264865bed4d6b7547ff199, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:50.274371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:50.275714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbe763-86a1-4cad-bc49-7a2d625f3aa8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:50.285405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 486/1000] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:50 2022] GET /identity/v3/endpoints?service_id=0f304ae3aa264865bed4d6b7547ff199&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:50.286747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:50.373854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:50.393012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f5ad-63dc-4659-a7b1-69b2fe810fcd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:50.428880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b3bb6-3e0a-428b-bdc2-ed62b84f03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:50.429553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b3bb6-3e0a-428b-bdc2-ed62b84f03ab None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.429844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b3bb6-3e0a-428b-bdc2-ed62b84f03ab None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.430127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b3bb6-3e0a-428b-bdc2-ed62b84f03ab None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.430812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b3bb6-3e0a-428b-bdc2-ed62b84f03ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:50.432176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b3bb6-3e0a-428b-bdc2-ed62b84f03ab None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:50.444783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 515/1001] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:50 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:23:50.763503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed28b0-0326-4d82-bf94-a923016a8098 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.764117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed28b0-0326-4d82-bf94-a923016a8098 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.764117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed28b0-0326-4d82-bf94-a923016a8098 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.765607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 487/1002] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:50.770780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053683-2104-447c-8e0f-f12a1f5b33ae None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.771194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053683-2104-447c-8e0f-f12a1f5b33ae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.771364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053683-2104-447c-8e0f-f12a1f5b33ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.788034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-91053683-2104-447c-8e0f-f12a1f5b33ae None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:50.789244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-91053683-2104-447c-8e0f-f12a1f5b33ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:50.858936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 516/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:50.882410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed28b0-0326-4d82-bf94-a923016a8098 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:50.969187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:50.971029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56f21f-e2e6-408c-93d6-961ab1359557 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.971519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56f21f-e2e6-408c-93d6-961ab1359557 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.971985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56f21f-e2e6-408c-93d6-961ab1359557 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.973565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 517/1004] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:50.979027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:50.988844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:50.988844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:50.989016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed28b0-0326-4d82-bf94-a923016a8098 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:50.997680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:50.999088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:51.023529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:51.024260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:51.024545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:51.024856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:51.025511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:51.026899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:51.039883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:51.041170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:51.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 488/1005] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Jul 18 10:23:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:51.046918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:51.088575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 518/1006] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:51.098865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:51.108864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:51.152201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdde3-5e46-453e-b2d5-cb8656aec588 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:51.189012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:51.189814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:51.190110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:51.190390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:51.191107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:51.191419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:51.192803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:51.198029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 489/1007] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:51.203429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:51.217224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb42fe8-5bf4-4e42-ad1f-e87e9332a7f6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:51.219207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:51.258838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:51.259515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:51.259802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:51.260082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:51.260842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:51.264227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:51.308941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5bb71-3541-4146-b804-294f55e674f4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:51.342222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:51.342884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:51.343173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:51.343452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:51.344254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:51.348874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:51.349066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 519/1008] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:51 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:51.349166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3a9c2-e9f2-4007-ab8b-871039339a67 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:51.362100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 490/1009] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Jul 18 10:23:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:51.363503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:51.440842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:51.490972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f5793-5372-4d06-b6e5-0f3f7d931e14 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:51.529683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ac90-7206-4c0b-aad3-69c10c61292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:51.530351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ac90-7206-4c0b-aad3-69c10c61292e None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:51.530638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ac90-7206-4c0b-aad3-69c10c61292e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:51.530916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ac90-7206-4c0b-aad3-69c10c61292e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:51.531655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ac90-7206-4c0b-aad3-69c10c61292e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:51.533052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ac90-7206-4c0b-aad3-69c10c61292e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:51.617944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 520/1010] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:52.535719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d1d7-9bb0-4ba1-80e6-a835c021ccf0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:52.536253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d1d7-9bb0-4ba1-80e6-a835c021ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:52.536253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d1d7-9bb0-4ba1-80e6-a835c021ccf0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:52.538285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 491/1011] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:52.544159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:52.544620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:52.544824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:52.561919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:52.563128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:52.642396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 521/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:52 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:52.665548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d1d7-9bb0-4ba1-80e6-a835c021ccf0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:52.746628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:52.772257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d1d7-9bb0-4ba1-80e6-a835c021ccf0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:52.806447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:52.807164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:52.807447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:52.807726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:52.808387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:52.809819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:52.823041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:52.824286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:23:52.825340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:52.833945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 492/1013] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:52.835799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:52.886483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:52.939424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718d290-b9a0-4ea8-8f0c-6eaad3a2c9a4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:52.974503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:52.975182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:52.975470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:52.975750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:52.976488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:52.976832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:52.978212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:52.988065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 522/1014] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:52.989319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.096375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce52fab-cbf2-4d99-8f34-283b89ac7daf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.132146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:53.132855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.133142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.133418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.134133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:53.134440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:53.135851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b26ea2-d0b8-4762-ab27-dd2f33ba5f76 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:53.145641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 493/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Jul 18 10:23:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:53.146793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.247041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed331eb-0994-4c48-acc9-308f3ad74197 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.268967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:53.290365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5da128-f5e8-492e-ad50-3c046e54541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:53.291065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5da128-f5e8-492e-ad50-3c046e54541d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.291359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5da128-f5e8-492e-ad50-3c046e54541d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.291635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5da128-f5e8-492e-ad50-3c046e54541d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.292375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5da128-f5e8-492e-ad50-3c046e54541d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:53.293775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5da128-f5e8-492e-ad50-3c046e54541d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:53.442201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 523/1016] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:53.578243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a977a3-c7e8-48d0-b5bd-23f334f77df1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.578768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a977a3-c7e8-48d0-b5bd-23f334f77df1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.578768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a977a3-c7e8-48d0-b5bd-23f334f77df1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.580328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 494/1017] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:53.586444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.586917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.587017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.605317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:53.606528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:53.684838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 524/1018] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:53.703149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07977720-815b-421a-95cb-0299a4a84d01 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.703385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07977720-815b-421a-95cb-0299a4a84d01 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.703646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07977720-815b-421a-95cb-0299a4a84d01 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.705190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 495/1019] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:53.707719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.711417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.712092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.712092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.719600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:53.732329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:53.733734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:53.805661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 496/1020] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:53.814813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:53.821835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1ca63-0efd-4bb9-a2d5-30a5bdc0e760 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.828554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.857063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:53.857865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.858158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.858441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.859098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:53.860477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:53.873334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:53.875609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:53.876471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 525/1021] 10.222.0.47 () {56 vars in 1105 bytes} [Mon Jul 18 10:23:53 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:53.880556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.941881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9a2fe-5540-41a6-994c-070980b0a638 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.983339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:53.984048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:53.984325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:53.984601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:53.985827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:53.987251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:53.989217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 497/1022] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:53.994636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:53.996720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21cee7-caeb-4021-9704-d2e2dd731a0c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:53.997346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:54.035742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:54.036407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:54.036713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:54.037794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:54.038567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:54.038889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:54.040249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:54.046048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 526/1023] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:54.051725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:54.076271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:54.105518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fd254-d20f-4494-98ea-f2ae54c6de4c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:54.142250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:54.143334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:54.143631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:54.143914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:54.144939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:54.147567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:54.155990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9490-3fe1-4209-93e6-f04e4dfdef1f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:54.191310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:54.191972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:54.192291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:54.192566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:54.193856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:54.194389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:54.195750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:54.200486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 527/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:54.206247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:54.291916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:54.306937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca531d22-2ba1-4bfb-bfd4-2e026a2b10c4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:54.339398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 498/1025] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:23:53 2022] POST /identity/v3/projects => generated 311 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:54.342348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:54.343111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:54.343471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:54.343817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:54.344731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] RBAC: Adding query filter params (interface=public, service_id=0fb631ca5b61460bbc302f83aa76c585, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:54.345126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:54.346638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8089d-38aa-4960-85f4-d835defdf44c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:54.357670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 528/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:23:54 2022] GET /identity/v3/endpoints?service_id=0fb631ca5b61460bbc302f83aa76c585&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:54.359414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:54.423035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:54.470217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460b75f-c4cd-4e31-8553-1743ca1d25ae None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:54.506490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa588b0-e9f5-429e-a57e-04b6bc008b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:54.507262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa588b0-e9f5-429e-a57e-04b6bc008b2c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:54.513047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa588b0-e9f5-429e-a57e-04b6bc008b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:54.513047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa588b0-e9f5-429e-a57e-04b6bc008b2c None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:54.513047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa588b0-e9f5-429e-a57e-04b6bc008b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:54.513047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa588b0-e9f5-429e-a57e-04b6bc008b2c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:54.517754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 499/1027] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:23:54 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:23:55.591903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eb5e4-48e5-4a90-a440-550c82de4ac0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:55.592383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eb5e4-48e5-4a90-a440-550c82de4ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:55.592383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eb5e4-48e5-4a90-a440-550c82de4ac0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:55.593841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 529/1028] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:55.598896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:55.599294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:55.600033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:55.617413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:55.618630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:55.689261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 500/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:55.712521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eb5e4-48e5-4a90-a440-550c82de4ac0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:55.814233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eb5e4-48e5-4a90-a440-550c82de4ac0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:55.848554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:55.849829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:55.850295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:55.850650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:55.860815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:55.869650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:55.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:23:55.871444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:55.878926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 530/1030] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Jul 18 10:23:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:55.879055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:55.925763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:55.981603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a61c22-554b-4dfa-b72d-459a218283b8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.016744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.017504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.017859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.018207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.019030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:56.019415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.020879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.036481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 501/1031] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:23:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:56.039551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.053915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:56.142759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c803-a571-4da5-bd68-132fa6dea7e2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.177036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.178581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.179028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.179413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.190795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.192362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:56.209722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:56.210800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:56.211881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:56.211881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:56.211881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:56.211881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:56.213391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 531/1032] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:56.218875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.335407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd893dcd-7f58-48b2-9915-f4ed86d0f76e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.345794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:56.370448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.371146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.371428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.371699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.372412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:56.372747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.374096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f671448-fcbc-41e8-9f9a-5fd237d235a4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.390361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 502/1033] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:56.391839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.398962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37818c6-34ea-4011-a12d-659b7b04bd72 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.399237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37818c6-34ea-4011-a12d-659b7b04bd72 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.399478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37818c6-34ea-4011-a12d-659b7b04bd72 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.401034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 503/1034] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:56.406070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.406789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.407446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.424875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:56.426154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:56.464539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:56.501039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 504/1035] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:56.509771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc2444-2b04-4f3e-abc2-ea1819300743 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.521999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.545854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.546653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.546945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.547226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.555016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.556466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d41a4e70-0b0d-4ddc-ba59-10d5278a75ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:23:56.570584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:56.571572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:56.573120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 532/1036] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:56.573755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c5cbf-e19b-4f75-bbb0-fadfcd494bdf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.573755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c5cbf-e19b-4f75-bbb0-fadfcd494bdf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.573895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c5cbf-e19b-4f75-bbb0-fadfcd494bdf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.575211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 533/1037] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:56.577307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c5cbf-e19b-4f75-bbb0-fadfcd494bdf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.621717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:56.624549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cad5df-af9c-41aa-bea4-b2a408230a5c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.657898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10b381-96dc-4183-9ba8-bb54187fc9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.658617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b381-96dc-4183-9ba8-bb54187fc9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.658899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b381-96dc-4183-9ba8-bb54187fc9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.659180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b381-96dc-4183-9ba8-bb54187fc9a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.660355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10b381-96dc-4183-9ba8-bb54187fc9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.661866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10b381-96dc-4183-9ba8-bb54187fc9a7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.664497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 505/1038] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:56.666156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.666956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.667238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.680746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c5cbf-e19b-4f75-bbb0-fadfcd494bdf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.683345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:56.684528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:56.703309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:56.715956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.716631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.716952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.717234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.718075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:56.718389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.719883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.737649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 534/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:56.738449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.751165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 506/1040] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:56.751994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:56.844541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fddd3c-87ae-4bd4-b76d-924178f1e9b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.850378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:56.854058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1439ad-fccb-4c08-9736-7e44a8bdc01d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:56.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.877725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.878006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.878280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.878983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.880423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.887843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:56.888484 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:56.889012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:56.889141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:56.955000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:56.956363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:56.965372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 535/1041] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:23:56 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:23:56.966188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.024489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.065858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.071192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 507/1042] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:23:57.077923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.098007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f77fe-3e53-4015-8e3e-65a77280790e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.140413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.141360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.141654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.141931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.142765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.148152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:57.158749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:57.159729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:23:57.160564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 536/1043] 10.222.0.47 () {56 vars in 1105 bytes} [Mon Jul 18 10:23:56 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:57.166031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.197871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14080b05-cc69-456a-94c8-4e5d41a35fa4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.234609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.235335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.235623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.235902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.247816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.251273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:23:57.271559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:57.272643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:57.278209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:23:57.278209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:23:57.278209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:23:57.278209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:57.278209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 508/1044] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:57.279375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.280013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.281914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85608951-25b0-4ec7-af2c-5679940c459f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.316898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.317672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.317966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.318764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.319822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:57.320475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.322386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.328065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 537/1045] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:57.333931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.335109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.388464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f84e9c-027e-470d-b0b4-2755ea7020ba None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.426869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.428075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.428559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.429055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.429794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:57.430247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.431622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.435608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23073abc-f57e-4920-b310-20a397d4f10f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.449041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 509/1046] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:57.450073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.475565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.476248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.476539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.476860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.477576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:57.477897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.479270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.484705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 538/1047] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:57.489786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.520416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.563228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79968a13-18cf-4e09-865a-65f1e039000d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.574738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.602260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.603058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.603343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.603560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a93abe-6526-4dc9-abdd-bb655811709c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.603663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.611848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.613714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:23:57.629032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:57.635496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 510/1048] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:57.636393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.643101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.643767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.644055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.644339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.645128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.646895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.739083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a68c94-cec3-41ec-930f-fc125fe845b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.749095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 539/1049] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:23:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:23:57.786393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.787806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.788165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.788445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.789309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:57.789616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.791087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be305eaf-cbd6-4a4f-9359-33acb7e63e32 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.810113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 511/1050] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:23:57.811544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:57.904844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:57.921176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bd1e9-fc69-4537-a537-b3205973b052 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:57.957982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:57.959294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:57.959703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:57.959991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:57.960921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] RBAC: Adding query filter params (role.id=ca9ad274f82348179efed0c66e246b84, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aa1f5d0bba8a4178977c19b7503ef6a0) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:57.961246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:57.962681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb16e2-5d80-4540-b4f7-b4d5d106e345 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:57.968461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 540/1051] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:23:57 2022] GET /identity/v3/role_assignments?user.id=aa1f5d0bba8a4178977c19b7503ef6a0&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ca9ad274f82348179efed0c66e246b84 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:59.091767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a3b48-b470-49e1-85f4-d88234264074 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.092289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a3b48-b470-49e1-85f4-d88234264074 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.092398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a3b48-b470-49e1-85f4-d88234264074 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.093883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 512/1052] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:59.098900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.099291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.099564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.116245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:59.117523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:59.187138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 541/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:23:59.211464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a3b48-b470-49e1-85f4-d88234264074 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:59.240364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:59.325190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a3b48-b470-49e1-85f4-d88234264074 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:59.407401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:59.408108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.408389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.408661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.420894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:23:59.429651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:23:59.430587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:23:59.431434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:23:59.439058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 513/1054] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:23:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:23:59.439198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:59.516745 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:59.538563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddc453-01f6-4509-8ffa-f1308b8c7fd2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:59.574737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:59.575412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.575702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.575981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.576773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:23:59.577090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:59.578508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:59.592603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 542/1055] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:59.593915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:59.696250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:59.713086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438031c6-bb45-4081-956a-4069e03cb274 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:59.747596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ced057-4d7b-4a2d-98e9-1f267ee862bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:59.748547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced057-4d7b-4a2d-98e9-1f267ee862bb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.748867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced057-4d7b-4a2d-98e9-1f267ee862bb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.749143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced057-4d7b-4a2d-98e9-1f267ee862bb None admin] PATH_INFO: `/v3/users/f893064b01d245ad8cde412bd4673e96` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.763152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ced057-4d7b-4a2d-98e9-1f267ee862bb None admin] RBAC: Authorizing `identity:get_user(user_id=f893064b01d245ad8cde412bd4673e96)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:59.764571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ced057-4d7b-4a2d-98e9-1f267ee862bb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:59.771007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 514/1056] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:23:59 2022] GET /identity/v3/users/f893064b01d245ad8cde412bd4673e96 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:23:59.772729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:59.808101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5589a4-b8e8-4063-bb28-f55821caaeb7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.808394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5589a4-b8e8-4063-bb28-f55821caaeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.808674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5589a4-b8e8-4063-bb28-f55821caaeb7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.810090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 515/1057] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:23:59.815183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.815578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.815962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.834377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:59.835842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:59.875543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ca423-6eb7-4881-bd1a-8c884623c5dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:59.914384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4df9e-9140-4983-8050-f26f5071b630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:59.915046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4df9e-9140-4983-8050-f26f5071b630 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:59.915479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 516/1058] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:23:59.918423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4df9e-9140-4983-8050-f26f5071b630 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:59.918423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4df9e-9140-4983-8050-f26f5071b630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:59.930154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:59.937374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:59.992439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4df9e-9140-4983-8050-f26f5071b630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:59.993882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4df9e-9140-4983-8050-f26f5071b630 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.011517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.036866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f252f-c581-45ff-942c-fc1ba9abec7c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.070917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.071560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.071842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.072114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.097192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 543/1059] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:00.100153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac377c-b346-48ef-9e0b-21c816a81145 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.100723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac377c-b346-48ef-9e0b-21c816a81145 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.100862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac377c-b346-48ef-9e0b-21c816a81145 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.102293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 544/1060] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:00.102996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac377c-b346-48ef-9e0b-21c816a81145 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.145167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.145678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.147744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.203786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac377c-b346-48ef-9e0b-21c816a81145 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.222219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.241370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ce70d-2422-4d97-8489-8e1a4e14e329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.242221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ce70d-2422-4d97-8489-8e1a4e14e329 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.242558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ce70d-2422-4d97-8489-8e1a4e14e329 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.242784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ce70d-2422-4d97-8489-8e1a4e14e329 None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.243954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ce70d-2422-4d97-8489-8e1a4e14e329 None admin] RBAC: Authorizing `identity:get_project(project_id=4ada6a03ce8948da86b5eb39cc597a2a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.245421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ce70d-2422-4d97-8489-8e1a4e14e329 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.247098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 545/1061] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:00.252808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.252964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 517/1062] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:23:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:24:00.253060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.253060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.253410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.272209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:00.273458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:00.340194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 546/1063] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:00.341437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.353757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525a2c0-c053-4331-b634-989ebe344cef None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.392478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.393172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.393458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.393735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.394156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.394609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a, user.id=f893064b01d245ad8cde412bd4673e96) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:00.394934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.397022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.402565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 518/1064] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/role_assignments?user.id=f893064b01d245ad8cde412bd4673e96&scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:00.403344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.448428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.464157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174af4-f8d4-4f47-bfed-89ecbd77667c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.506317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.507192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.507485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.507768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.519175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.519566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb28418-0cc5-4568-bf14-693a5cb9cb9c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.520665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:00.541114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:00.542103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:00.543033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:24:00.543033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:00.543033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:00.543033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:00.547700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 547/1065] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:00.548647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.561073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.562111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.562223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.562427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.574290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:00.582794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:00.584395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:24:00.585893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 519/1066] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:00.590522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.649513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.659695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c312-8ed8-4036-a29c-2aa672deddb0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.694591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:00.696460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.697167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.697461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.697746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.697904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68408b5f-2fe6-4b74-a428-c5a39d0b5bdf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.698532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:00.698846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.700859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.713547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 548/1067] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:00.734511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.735588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.736080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.736565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.737641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:00.738123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.739692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.750744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 520/1068] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:00.757742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:00.871428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7ef7-8ad2-42b6-956e-6915f8f1b691 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:00.907944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:00.908576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:00.908893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:00.909174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:00.922750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:00.924215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:00.942327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:00.943873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:00.946094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:24:00.946094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:00.946094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:00.946094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:00.946094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 549/1069] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:00.951304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.059240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34eef-1bb2-44d8-b72b-16f9d1def764 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.093262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.093921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.094207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.094488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.095221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:01.095534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:01.096922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:01.107990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 521/1070] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:01.113648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.117139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:01.214094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc2112-9453-48df-8035-98f16586eb3d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.251030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.253426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.253426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.253426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.260427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:01.261941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:24:01.277333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:01.278527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:01.279742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 550/1071] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:01.284314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.307470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:01.384752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c258b-1fb4-4bb4-9257-98606956f8cb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.418105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.418767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.419055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.419342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.420178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:01.420494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:01.421948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:01.439985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 522/1072] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:01.446073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.509009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:01.547309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2d392-648f-4dba-b5e1-fd6812926438 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.580341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.581167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.581458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.581739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.593216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:01.594663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:01.611969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:01.612898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:01.613775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:24:01.613775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:01.613775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:01.613775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:01.613775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 551/1073] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:01.619631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.699043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:01.720849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125488f4-32ab-4805-8bc1-36b72568abd6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.754172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.754835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.755126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.755457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.756271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:01.756587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:01.757971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:01.769089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 523/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:01.775522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:01.873063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4bd18-82af-4156-b17e-955fb7bc3076 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:01.886254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:01.905958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:01.906904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:01.907189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:01.907469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:01.914106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:01.915549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:24:01.930288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:01.931157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:01.932047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 552/1075] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:01.938621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.040230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895be1-f9a4-49bd-9233-47dc61f9be4a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.075671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.076327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.076609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.076914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.077711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:02.078024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.079835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f0419-fe99-4778-8088-ad6c8a6fefc7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.087669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.092930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 524/1076] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:02.098914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.200155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3f02b-c1a2-481e-a452-4627e05418a8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.233655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3927a8-f7ad-41f5-863b-9792a2ee597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.234538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3927a8-f7ad-41f5-863b-9792a2ee597d None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.234823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3927a8-f7ad-41f5-863b-9792a2ee597d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.235098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3927a8-f7ad-41f5-863b-9792a2ee597d None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aa1f5d0bba8a4178977c19b7503ef6a0/roles/ca9ad274f82348179efed0c66e246b84` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.237303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3927a8-f7ad-41f5-863b-9792a2ee597d None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=aa1f5d0bba8a4178977c19b7503ef6a0, role_id=ca9ad274f82348179efed0c66e246b84)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.238731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3927a8-f7ad-41f5-863b-9792a2ee597d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.326975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 553/1077] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:02 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/aa1f5d0bba8a4178977c19b7503ef6a0/roles/ca9ad274f82348179efed0c66e246b84 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:02.414054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8b850-e1db-4a97-8474-4fecf13c1e23 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.414587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8b850-e1db-4a97-8474-4fecf13c1e23 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.414587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8b850-e1db-4a97-8474-4fecf13c1e23 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.415992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 525/1078] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:02.421736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b819-9401-4395-a828-3443f9d65110 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.422229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b819-9401-4395-a828-3443f9d65110 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.422580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b819-9401-4395-a828-3443f9d65110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.441048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8826b819-9401-4395-a828-3443f9d65110 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:02.442280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8826b819-9401-4395-a828-3443f9d65110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:02.474572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.513698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 554/1079] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:02.536161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8b850-e1db-4a97-8474-4fecf13c1e23 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.636419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8b850-e1db-4a97-8474-4fecf13c1e23 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.669478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.670262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.670539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.670807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.681398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:02.689004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:02.690091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:24:02.690970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 526/1080] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:02.697181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b819-9401-4395-a828-3443f9d65110 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.795208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b819-9401-4395-a828-3443f9d65110 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.828374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.829074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.829364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.829646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.830394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:02.830708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.832080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.840513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 555/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:02.846463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:02.904351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:02.945876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf3a4a-a732-4fb6-90d6-05f5357313de None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:02.979395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:02.980257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:02.980544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:02.980857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] PATH_INFO: `/v3/users/f893064b01d245ad8cde412bd4673e96` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:02.982046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=f893064b01d245ad8cde412bd4673e96)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:02.983467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:02.985166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 527/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/users/f893064b01d245ad8cde412bd4673e96 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:02.991486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:03.018888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:03.091985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ada313-d4e6-4448-ba83-e237a96e9a4a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.126451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5eb3d-ba5f-409c-8922-5e00b2c587f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.127302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5eb3d-ba5f-409c-8922-5e00b2c587f3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.127588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5eb3d-ba5f-409c-8922-5e00b2c587f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.127866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5eb3d-ba5f-409c-8922-5e00b2c587f3 None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.129056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5eb3d-ba5f-409c-8922-5e00b2c587f3 None admin] RBAC: Authorizing `identity:get_project(project_id=4ada6a03ce8948da86b5eb39cc597a2a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.130498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5eb3d-ba5f-409c-8922-5e00b2c587f3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.132367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 556/1083] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:02 2022] GET /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:03.138293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:03.236706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473b1ab-0df9-43d0-8af2-0bde5584b6a4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:03.263959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:03.273322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baeedb-2668-4570-9de4-9aec75aa920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:03.274215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baeedb-2668-4570-9de4-9aec75aa920e None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:03.274503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baeedb-2668-4570-9de4-9aec75aa920e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:03.276261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baeedb-2668-4570-9de4-9aec75aa920e None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a/users/f893064b01d245ad8cde412bd4673e96/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:03.276931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baeedb-2668-4570-9de4-9aec75aa920e None admin] RBAC: Authorizing `identity:create_grant(project_id=4ada6a03ce8948da86b5eb39cc597a2a, user_id=f893064b01d245ad8cde412bd4673e96, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:03.278364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baeedb-2668-4570-9de4-9aec75aa920e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:03.423842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 528/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:03 2022] PUT /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a/users/f893064b01d245ad8cde412bd4673e96/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:04.317782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369cf6a-6e01-452b-946e-30b496b7f682 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:04.318326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369cf6a-6e01-452b-946e-30b496b7f682 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:04.318326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369cf6a-6e01-452b-946e-30b496b7f682 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:04.319878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 557/1085] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:04.325604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:04.325901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:04.326190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:04.343807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:04.345033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:04.409879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 529/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:04.437539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369cf6a-6e01-452b-946e-30b496b7f682 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:04.530820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:04.538777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369cf6a-6e01-452b-946e-30b496b7f682 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:04.572667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:04.573501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:04.573789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:04.574067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:04.584241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:04.592538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:04.593566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:24:04.594487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:04.596961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 558/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Jul 18 10:24:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:04.600533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:04.696524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f1bd6-2948-4660-979e-9d8c9f9bab0e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:04.716479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:04.734300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:04.734998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:04.735278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:04.735555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:04.736303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:04.736614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:04.738002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:04.746052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 530/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:24:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:04.751722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:04.851383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55050152-b82a-4627-8157-e4e72f6e6bcf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:04.883616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:04.884436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:04.884760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:04.885024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:04.895380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:04.897398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:04.914676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:04.915655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:04.916558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:24:04.916558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:04.916558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:04.916558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:04.916558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 559/1089] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:04.921597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.018537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18906bb-6bd9-4ca6-a502-67dff2bc4573 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.051877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.052559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.052870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.053148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.053874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:05.054182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:05.055538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:05.067123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 531/1090] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:05.073256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.083032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.172671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96989c02-3586-4397-b3eb-018398986d8a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.205388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.206176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.206459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.206736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.215088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:05.216520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:24:05.230233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:05.231406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:05.232609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 560/1091] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:05.237221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.282686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.341796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b254f72-ea1e-4c4e-bd2f-67836805b945 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.376907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.377571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.377857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.378136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.378977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:05.379288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:05.381460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcc7d6-a6dc-4f8e-94ec-6e3ca4443e0e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:05.399136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 532/1092] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:05.405043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.437456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e298a-2f32-4a20-96f0-9e0d091ae79d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.437670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e298a-2f32-4a20-96f0-9e0d091ae79d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.437948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e298a-2f32-4a20-96f0-9e0d091ae79d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.439316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 533/1093] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:05.445454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.445454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.445776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.462385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:05.463579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:05.470107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.504422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b80fed-29c9-417f-bbd4-94a0032c96c0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.529945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 534/1094] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:24:05.539143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.539792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.540080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.540358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.553244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.579918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.607700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:05.609238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:05.652234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4d609-ed54-4623-9e92-8ea825e1353f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.674732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.685814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.686604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.686891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.687996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.697352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:05.704993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:05.705912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:24:05.706924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 535/1095] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:05.709601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 561/1096] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:05.713172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.715002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.796586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.811462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eefdd0-0253-4f76-b3de-825f996e3e69 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.813048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15832eb-c020-494d-9700-2775a9458480 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.845148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.845939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.846108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.846220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.846502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.847045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.847713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.847981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:05.848756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:05.849608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:05.850968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:05.857063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:05.858529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:05.859751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 536/1097] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:05.865761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:05.876090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:05.877188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:05.878082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:24:05.878082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:05.878082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:24:05.878082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:05.878082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 562/1098] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:05.883460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:05.902977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:05.964720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388eccd-894c-46ad-b4d1-ecb3a0720f5d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.985158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16055b79-9bf3-47f2-9826-bb5fb2c42250 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:05.998235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:05.999107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:05.999397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:05.999676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] PATH_INFO: `/v3/users/f893064b01d245ad8cde412bd4673e96` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.000915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] RBAC: Authorizing `identity:get_user(user_id=f893064b01d245ad8cde412bd4673e96)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.002349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.003991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 537/1099] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/users/f893064b01d245ad8cde412bd4673e96 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.009673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.019137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.019797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.020082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.020380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.021165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:06.021632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.022998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.026797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.034362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 563/1100] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.040344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.107372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c4ab2-39ab-4769-9648-3a6b8f615576 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.119030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.139097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08d453-7676-4d41-bb48-15894f21cc28 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.141004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.141654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.141936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.142215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.172788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.173578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.173855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.174138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.180908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.182374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:24:06.197155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:06.198092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:06.199084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 564/1101] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:24:06.204188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.214937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.216309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.233747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.305013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ecc9b-a44e-4d3b-8111-6bc13ae62e13 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.317296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 538/1102] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:06.323020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.339547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.340204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.340493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.340807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.341651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:06.341965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.343390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.350520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.355528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 565/1103] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.361544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.428098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d90ab-6604-4aeb-ba9c-68685bb08411 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.444574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:06.462524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.463466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.463754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.464035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.465307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] RBAC: Authorizing `identity:get_project(project_id=4ada6a03ce8948da86b5eb39cc597a2a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.465587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bd91b-dbb3-4ae4-9242-d86ad7451731 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.466688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.468462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 539/1104] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:06.475609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:06.499670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.500328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.500615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.500937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.501762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] RBAC: Adding query filter params (role.id=ca9ad274f82348179efed0c66e246b84, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=aa1f5d0bba8a4178977c19b7503ef6a0) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:06.502079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.503507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c253-bbc4-4843-8df7-841848c76348 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.509882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 566/1105] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/role_assignments?user.id=aa1f5d0bba8a4178977c19b7503ef6a0&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ca9ad274f82348179efed0c66e246b84 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:06.585610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fccb9-15ac-4d23-b619-b2158c65e30b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:06.620982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:06.621656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:06.621941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:06.622222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:06.623072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a, user.id=f893064b01d245ad8cde412bd4673e96) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:06.623389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:06.624881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285aed-0b04-4119-b2eb-510d41475fd9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:06.644784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 540/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:06 2022] GET /identity/v3/role_assignments?user.id=f893064b01d245ad8cde412bd4673e96&scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:24:08.672414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af230ada-345b-4ee3-b06d-55a5fc831eec None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:08.672971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af230ada-345b-4ee3-b06d-55a5fc831eec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:08.672971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af230ada-345b-4ee3-b06d-55a5fc831eec None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:08.674296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 567/1107] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:08.680198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad36c-52b3-463e-b869-1b821332311b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:08.680627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad36c-52b3-463e-b869-1b821332311b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:08.680785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad36c-52b3-463e-b869-1b821332311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:08.697755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7daad36c-52b3-463e-b869-1b821332311b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:08.698926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7daad36c-52b3-463e-b869-1b821332311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:08.742093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:08.764092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 541/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:08.820567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af230ada-345b-4ee3-b06d-55a5fc831eec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:08.856430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:08.896545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af230ada-345b-4ee3-b06d-55a5fc831eec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:08.928978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bbb6a-971d-4c9c-b3f5-f0b78528d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:08.929700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bbb6a-971d-4c9c-b3f5-f0b78528d819 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:08.929987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bbb6a-971d-4c9c-b3f5-f0b78528d819 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:08.930266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bbb6a-971d-4c9c-b3f5-f0b78528d819 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:08.931439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bbb6a-971d-4c9c-b3f5-f0b78528d819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:08.933057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bbb6a-971d-4c9c-b3f5-f0b78528d819 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:08.934658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 568/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:08.940523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daad36c-52b3-463e-b869-1b821332311b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:09.037976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daad36c-52b3-463e-b869-1b821332311b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:09.070877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87234d-957a-49c5-b004-3f9b039a11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:09.071530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87234d-957a-49c5-b004-3f9b039a11f5 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:09.071818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87234d-957a-49c5-b004-3f9b039a11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:09.072098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87234d-957a-49c5-b004-3f9b039a11f5 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:09.072995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87234d-957a-49c5-b004-3f9b039a11f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:09.074430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87234d-957a-49c5-b004-3f9b039a11f5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:09.210246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 542/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:08 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:11.264232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1eae71-cc40-4d73-ba77-abaefb08f052 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:11.264799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1eae71-cc40-4d73-ba77-abaefb08f052 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:11.264896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1eae71-cc40-4d73-ba77-abaefb08f052 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:11.266181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 569/1111] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:11.272339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:11.272811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:11.272921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:11.289317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:11.290496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:11.316282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:11.356714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 543/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:11.379865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1eae71-cc40-4d73-ba77-abaefb08f052 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:11.481423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1eae71-cc40-4d73-ba77-abaefb08f052 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:11.508651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:11.514121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:11.519097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:11.519097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:11.519097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] PATH_INFO: `/v3/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:11.519097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] RBAC: Authorizing `identity:get_role(role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:11.519097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:11.519799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 570/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:11 2022] GET /identity/v3/roles/6413d71218d344e8b26c8577cac9c4dc => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:11.526414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:11.623500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb77c40-7a52-4eac-bfc8-ef86703c7d1c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:11.656255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:11.657140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:11.657430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:11.657711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] PATH_INFO: `/v3/users/319f1686b6ee490ca733845bd9bfa770` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:11.671113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] RBAC: Authorizing `identity:get_user(user_id=319f1686b6ee490ca733845bd9bfa770)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:11.672545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:11.674200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 544/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:11 2022] GET /identity/v3/users/319f1686b6ee490ca733845bd9bfa770 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:11.680637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:11.778184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d20cb-709b-42bb-9eb3-e37edd1f326c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:11.811247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:11.811878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:11.812162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:11.812436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:11.873799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:11.877144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:11.878614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:11.976806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 571/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:11.982678 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:12.007245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:12.079151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98a797-b2f6-4481-b592-6ad73ef20240 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:12.112552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309d983-9b6e-4f01-b003-23b277fa133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:12.113441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309d983-9b6e-4f01-b003-23b277fa133c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.113724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309d983-9b6e-4f01-b003-23b277fa133c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.113996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309d983-9b6e-4f01-b003-23b277fa133c None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.115758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309d983-9b6e-4f01-b003-23b277fa133c None admin] RBAC: Authorizing `identity:get_project(project_id=4ada6a03ce8948da86b5eb39cc597a2a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:12.117238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309d983-9b6e-4f01-b003-23b277fa133c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:12.118938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 545/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:11 2022] GET /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:12.125172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:12.219186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:12.222994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725318d-53e0-4f6c-9d60-177c24b0a4d8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:12.256737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:12.257611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:12.258077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:12.258561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:12.259607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] RBAC: Adding query filter params (role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a, user.id=319f1686b6ee490ca733845bd9bfa770) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:12.259931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:12.261404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7c1f0-283a-4e4f-b9f8-6db73d930e17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:12.271585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 572/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:12 2022] GET /identity/v3/role_assignments?user.id=319f1686b6ee490ca733845bd9bfa770&scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.242500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071782ef-f433-4a6b-ae77-10e181ef5786 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.243032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071782ef-f433-4a6b-ae77-10e181ef5786 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.243124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071782ef-f433-4a6b-ae77-10e181ef5786 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.244431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 546/1118] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:14.249748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.250171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.250308 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.267022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:14.268202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:14.336812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 573/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:14.358787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071782ef-f433-4a6b-ae77-10e181ef5786 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.448361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.459332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071782ef-f433-4a6b-ae77-10e181ef5786 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.492623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.493442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.493729 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.494005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] PATH_INFO: `/v3/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.495485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] RBAC: Authorizing `identity:get_role(role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.496875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.498239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 547/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/roles/6413d71218d344e8b26c8577cac9c4dc => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.504082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.593107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:14.602144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09fcda-1fff-44ee-9ad8-e26a410476b3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.635835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.636719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.637012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.637332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] PATH_INFO: `/v3/users/319f1686b6ee490ca733845bd9bfa770` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.639207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] RBAC: Authorizing `identity:get_user(user_id=319f1686b6ee490ca733845bd9bfa770)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.640641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.642295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 574/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/users/319f1686b6ee490ca733845bd9bfa770 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.648194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.744340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729ca4f-aa38-49fd-abd8-9e813ef6f8b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.778126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b36799-17fb-4e92-9d1c-63c8cd2d6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.779014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b36799-17fb-4e92-9d1c-63c8cd2d6238 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.779306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b36799-17fb-4e92-9d1c-63c8cd2d6238 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.779598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b36799-17fb-4e92-9d1c-63c8cd2d6238 None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.780905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b36799-17fb-4e92-9d1c-63c8cd2d6238 None admin] RBAC: Authorizing `identity:get_project(project_id=4ada6a03ce8948da86b5eb39cc597a2a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.782362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b36799-17fb-4e92-9d1c-63c8cd2d6238 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:14.784271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 548/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:14 2022] GET /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:14.790748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:14.887225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d7f74-387a-40d1-ab61-3ed0229f259e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:14.919683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce403-4680-46bb-9879-de567e4a4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:14.920560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce403-4680-46bb-9879-de567e4a4ad9 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:14.920878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce403-4680-46bb-9879-de567e4a4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:14.921154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce403-4680-46bb-9879-de567e4a4ad9 None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a/users/319f1686b6ee490ca733845bd9bfa770/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:14.923331 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce403-4680-46bb-9879-de567e4a4ad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ada6a03ce8948da86b5eb39cc597a2a, user_id=319f1686b6ee490ca733845bd9bfa770, role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:14.924782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce403-4680-46bb-9879-de567e4a4ad9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:15.023551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 575/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:14 2022] PUT /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a/users/319f1686b6ee490ca733845bd9bfa770/roles/6413d71218d344e8b26c8577cac9c4dc => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:17.011758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b37d-1556-474f-ba0b-7dffc60af596 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.012288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b37d-1556-474f-ba0b-7dffc60af596 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.012405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b37d-1556-474f-ba0b-7dffc60af596 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.013733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 549/1124] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:17.018972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.019381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.019543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.037365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:17.038551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:17.052214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:17.103474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 576/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:17.126898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394b37d-1556-474f-ba0b-7dffc60af596 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:17.154295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:17.227597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394b37d-1556-474f-ba0b-7dffc60af596 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:17.261063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:17.261971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.262199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.262474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] PATH_INFO: `/v3/roles/6413d71218d344e8b26c8577cac9c4dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.263967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] RBAC: Authorizing `identity:get_role(role_id=6413d71218d344e8b26c8577cac9c4dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:17.265360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:17.266740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 550/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:17 2022] GET /identity/v3/roles/6413d71218d344e8b26c8577cac9c4dc => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:17.273356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:17.368408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda98e1-97a6-4338-b0b8-061fd70423b1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:17.401384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:17.402227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.402510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.402784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] PATH_INFO: `/v3/users/319f1686b6ee490ca733845bd9bfa770` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.403958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] RBAC: Authorizing `identity:get_user(user_id=319f1686b6ee490ca733845bd9bfa770)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:17.405395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:17.406996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 577/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:17 2022] GET /identity/v3/users/319f1686b6ee490ca733845bd9bfa770 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:17.412799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:17.503709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:17.511111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ff1e-09ce-489b-94bf-9e7e85bf1bd7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:17.543914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:17.544545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.544861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.545137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.612015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:17.613479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:17.701247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:17.708851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 551/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:17.714998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:17.735059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:17.812347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4fbfa-ad9e-4c20-b714-0fe9a9def017 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:17.845577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032b781-a7a9-49f6-bf69-0f91ad6318ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:17.846434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032b781-a7a9-49f6-bf69-0f91ad6318ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.846718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032b781-a7a9-49f6-bf69-0f91ad6318ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.846993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032b781-a7a9-49f6-bf69-0f91ad6318ce None admin] PATH_INFO: `/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.848148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032b781-a7a9-49f6-bf69-0f91ad6318ce None admin] RBAC: Authorizing `identity:get_project(project_id=4ada6a03ce8948da86b5eb39cc597a2a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:17.849652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032b781-a7a9-49f6-bf69-0f91ad6318ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:17.851304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 578/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:17 2022] GET /identity/v3/projects/4ada6a03ce8948da86b5eb39cc597a2a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:17.857377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:17.959280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16379c8-275a-4aae-b70b-8206ed69c082 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:17.992483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:17.993168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:17.993459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:17.993736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:17.994573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] RBAC: Adding query filter params (role.id=6413d71218d344e8b26c8577cac9c4dc, scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a, user.id=319f1686b6ee490ca733845bd9bfa770) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:17.994888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:17.996303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c586a-0faf-489d-a551-1bf8de48f5c3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:18.002201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 552/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:17 2022] GET /identity/v3/role_assignments?user.id=319f1686b6ee490ca733845bd9bfa770&scope.project.id=4ada6a03ce8948da86b5eb39cc597a2a&role.id=6413d71218d344e8b26c8577cac9c4dc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.005399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5061e-d78f-4209-911f-cd1657302e3b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.005964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5061e-d78f-4209-911f-cd1657302e3b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.005964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5061e-d78f-4209-911f-cd1657302e3b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.007317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 579/1131] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:20.012532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.012935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.013122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.033665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:20.034859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:20.088410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.101470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 553/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:20.124860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5061e-d78f-4209-911f-cd1657302e3b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.207887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:20.226330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5061e-d78f-4209-911f-cd1657302e3b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.260254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefac99-a1d0-4fd2-aac9-785e82cf6d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.261009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefac99-a1d0-4fd2-aac9-785e82cf6d95 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.261299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefac99-a1d0-4fd2-aac9-785e82cf6d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.261581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefac99-a1d0-4fd2-aac9-785e82cf6d95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.262757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefac99-a1d0-4fd2-aac9-785e82cf6d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.264189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefac99-a1d0-4fd2-aac9-785e82cf6d95 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.265936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 580/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:20.271841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:20.370703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49980a15-5ce5-406e-a639-9ea808794cb7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:20.403126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77d799-482f-4e55-b605-4e3f6feaeba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:20.403772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77d799-482f-4e55-b605-4e3f6feaeba2 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:20.404057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77d799-482f-4e55-b605-4e3f6feaeba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:20.404333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77d799-482f-4e55-b605-4e3f6feaeba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:20.405068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77d799-482f-4e55-b605-4e3f6feaeba2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:20.406485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77d799-482f-4e55-b605-4e3f6feaeba2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:20.492212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 554/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:24:20 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:22.537984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fce604-c6a3-4510-bea3-8eb9dd675664 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.538526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fce604-c6a3-4510-bea3-8eb9dd675664 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.538526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fce604-c6a3-4510-bea3-8eb9dd675664 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.539825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 581/1135] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:22.544906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.545345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.545471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.562211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:22.563381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:22.606316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:22.644359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 555/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:22.669836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fce604-c6a3-4510-bea3-8eb9dd675664 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:22.768366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fce604-c6a3-4510-bea3-8eb9dd675664 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:22.800472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d2ea-5971-485f-bd90-4fa9811b71c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:22.801217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d2ea-5971-485f-bd90-4fa9811b71c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.802143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d2ea-5971-485f-bd90-4fa9811b71c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.802558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d2ea-5971-485f-bd90-4fa9811b71c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.803730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85d2ea-5971-485f-bd90-4fa9811b71c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:22.805185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85d2ea-5971-485f-bd90-4fa9811b71c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:22.806757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 582/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:24:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:22.812572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:22.907608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38693b-0e24-4c5c-a515-cc1a756e8fbc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:22.939990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e4034-205b-4d53-b38f-9b16c801753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:22.940639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e4034-205b-4d53-b38f-9b16c801753f None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:22.940944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e4034-205b-4d53-b38f-9b16c801753f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:22.941220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e4034-205b-4d53-b38f-9b16c801753f None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:22.941951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e4034-205b-4d53-b38f-9b16c801753f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:22.944005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e4034-205b-4d53-b38f-9b16c801753f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:23.028364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 556/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:22 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:24:25.055456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1e83a-311b-47cb-a564-347d041eb4ec None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.055988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1e83a-311b-47cb-a564-347d041eb4ec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.055988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1e83a-311b-47cb-a564-347d041eb4ec None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.057395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 583/1139] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:25.062578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.063013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.063141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.080902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:25.082336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:25.102331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.148182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 557/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:25.170896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1e83a-311b-47cb-a564-347d041eb4ec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.195742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.274556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1e83a-311b-47cb-a564-347d041eb4ec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.308278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.309039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.309324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.309603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.319588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:25.327188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:25.328209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:25.329085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 584/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:25.335010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.432099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b015b-17a0-41a3-879e-9497f43746b0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.464635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.465376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.465661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.465939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.466686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:25.466996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.468351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.476838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 558/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:25.482642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.562051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.578691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641545e-6825-4bdd-96f7-862268ecd10d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.610835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.614498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.614498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.614498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] PATH_INFO: `/v3/users/a0f124ad0aeb44508cf8af1e3dc7bbb4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.625809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] RBAC: Authorizing `identity:get_user(user_id=a0f124ad0aeb44508cf8af1e3dc7bbb4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.627252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.628899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 585/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/users/a0f124ad0aeb44508cf8af1e3dc7bbb4 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:25.634733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:25.653561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.733120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c857ba-661a-4e37-a2ef-3ff0ab3c49e3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:25.765009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:25.765647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:25.765934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:25.766218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:25.830231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:25.831592 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:25.849490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:25.925875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 559/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:24:25.931790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.028308 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d214ec-2884-466b-b0ed-ed2fc007a20d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.061084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0e7fa-6cdf-432d-ad28-3e243c6eea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.061937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e7fa-6cdf-432d-ad28-3e243c6eea0e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.062220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e7fa-6cdf-432d-ad28-3e243c6eea0e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.062495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e7fa-6cdf-432d-ad28-3e243c6eea0e None admin] PATH_INFO: `/v3/projects/dd3f755b7ca242658541b979d3a98edd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.063658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0e7fa-6cdf-432d-ad28-3e243c6eea0e None admin] RBAC: Authorizing `identity:get_project(project_id=dd3f755b7ca242658541b979d3a98edd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:26.065110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0e7fa-6cdf-432d-ad28-3e243c6eea0e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:26.067299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 586/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:25 2022] GET /identity/v3/projects/dd3f755b7ca242658541b979d3a98edd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:26.073527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:26.170837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c41ef-7fc5-4da4-b246-69e9bf13fed0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:26.186993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:26.202983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:26.203619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:26.203901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:26.204174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:26.205066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=dd3f755b7ca242658541b979d3a98edd, user.id=a0f124ad0aeb44508cf8af1e3dc7bbb4) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:26.215553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:26.215553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab445f-4723-4361-9498-38aa385aabeb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:26.215553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 560/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:26 2022] GET /identity/v3/role_assignments?user.id=a0f124ad0aeb44508cf8af1e3dc7bbb4&scope.project.id=dd3f755b7ca242658541b979d3a98edd&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.217835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d8bd-2d62-453c-8e27-44e9191e35e0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.218391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d8bd-2d62-453c-8e27-44e9191e35e0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.218391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d8bd-2d62-453c-8e27-44e9191e35e0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.219765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 587/1147] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:28.225795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.226185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.226383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.242772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:28.243949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:28.309119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 561/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:28.332984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011d8bd-2d62-453c-8e27-44e9191e35e0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.343733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.431225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011d8bd-2d62-453c-8e27-44e9191e35e0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.465360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.466148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.466435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.466710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.476532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:28.484070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:28.485164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:28.486338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 588/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:28.491414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.587229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe12db2-b6ac-4e4b-95ac-140371254deb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.599775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.619961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.620699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.620987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.621264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.622010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:28.622315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.623660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.632378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 562/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.638147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.702699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:28.735230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7798ec-6a64-4337-8f68-1d3dfa7be414 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.767630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.768489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.768808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.769090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] PATH_INFO: `/v3/users/a0f124ad0aeb44508cf8af1e3dc7bbb4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.770889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] RBAC: Authorizing `identity:get_user(user_id=a0f124ad0aeb44508cf8af1e3dc7bbb4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.772313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.773952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 589/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/users/a0f124ad0aeb44508cf8af1e3dc7bbb4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.779677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:28.875603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d139866-cefd-44ed-a387-752959b45008 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:28.907713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58455227-adcb-4df1-95f4-1ad4757da679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:28.908575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58455227-adcb-4df1-95f4-1ad4757da679 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:28.908881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58455227-adcb-4df1-95f4-1ad4757da679 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:28.909155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58455227-adcb-4df1-95f4-1ad4757da679 None admin] PATH_INFO: `/v3/projects/dd3f755b7ca242658541b979d3a98edd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:28.910314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58455227-adcb-4df1-95f4-1ad4757da679 None admin] RBAC: Authorizing `identity:get_project(project_id=dd3f755b7ca242658541b979d3a98edd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:28.911919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58455227-adcb-4df1-95f4-1ad4757da679 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:28.913633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 563/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:28 2022] GET /identity/v3/projects/dd3f755b7ca242658541b979d3a98edd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:28.919558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:29.016233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c7ba5-9a71-47d9-bd39-a022ebea66f4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:29.048617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1313c-213b-4f72-ab45-32111d35fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:29.049523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1313c-213b-4f72-ab45-32111d35fc75 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:29.049811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1313c-213b-4f72-ab45-32111d35fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:29.050095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1313c-213b-4f72-ab45-32111d35fc75 None admin] PATH_INFO: `/v3/projects/dd3f755b7ca242658541b979d3a98edd/users/a0f124ad0aeb44508cf8af1e3dc7bbb4/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:29.052134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1313c-213b-4f72-ab45-32111d35fc75 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd3f755b7ca242658541b979d3a98edd, user_id=a0f124ad0aeb44508cf8af1e3dc7bbb4, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:29.053582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1313c-213b-4f72-ab45-32111d35fc75 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:29.056527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:29.146908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 590/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:28 2022] PUT /identity/v3/projects/dd3f755b7ca242658541b979d3a98edd/users/a0f124ad0aeb44508cf8af1e3dc7bbb4/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:31.122616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c772-9643-4c52-931e-65122cfd6c25 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.123151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c772-9643-4c52-931e-65122cfd6c25 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.123151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c772-9643-4c52-931e-65122cfd6c25 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.124547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 564/1154] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:31.130234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.130687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.130877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.147048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:31.148220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:31.213204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 591/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:31.237133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5c772-9643-4c52-931e-65122cfd6c25 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.281444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.338275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5c772-9643-4c52-931e-65122cfd6c25 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.370064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.370848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.371141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.371417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.382377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:31.389938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:31.391019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:24:31.392019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 565/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:31.398401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.502414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48d923-b2ee-4c9c-a77d-a4a37284eef8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.528736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.536907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.537573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.537864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.538148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.538896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:31.539211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.540573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.548400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 592/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:31.554418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.651777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc89f74-1dd6-447d-a858-59c04816b735 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.653233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.685965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.686821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.687102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.687374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] PATH_INFO: `/v3/users/a0f124ad0aeb44508cf8af1e3dc7bbb4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.688550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] RBAC: Authorizing `identity:get_user(user_id=a0f124ad0aeb44508cf8af1e3dc7bbb4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.689998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.691811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 566/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/users/a0f124ad0aeb44508cf8af1e3dc7bbb4 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:31.697584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:31.795078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f1bf-8709-48cd-9fbd-4841a5aa81c4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:31.826830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:31.827464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:31.827753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:31.828028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:31.881309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:31.892919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:31.897083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:31.990063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 593/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:31.995768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.092602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f68028-2e59-42b9-8498-ce3fbac736e5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.124976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b495bcb-12e4-42e8-85c0-dafd1d1f34f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.125813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b495bcb-12e4-42e8-85c0-dafd1d1f34f3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.126097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b495bcb-12e4-42e8-85c0-dafd1d1f34f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.126374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b495bcb-12e4-42e8-85c0-dafd1d1f34f3 None admin] PATH_INFO: `/v3/projects/dd3f755b7ca242658541b979d3a98edd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.127529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b495bcb-12e4-42e8-85c0-dafd1d1f34f3 None admin] RBAC: Authorizing `identity:get_project(project_id=dd3f755b7ca242658541b979d3a98edd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.128959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b495bcb-12e4-42e8-85c0-dafd1d1f34f3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.131318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 567/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:31 2022] GET /identity/v3/projects/dd3f755b7ca242658541b979d3a98edd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:32.137365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:32.221713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:32.233099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0596d3-ce38-44af-bfa0-25c1ecf70170 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:32.265932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:32.266583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:32.266865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:32.267140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:32.267960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=dd3f755b7ca242658541b979d3a98edd, user.id=a0f124ad0aeb44508cf8af1e3dc7bbb4) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:32.268269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:32.269716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e6726-a910-4351-9218-03dc36cca279 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:32.276162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 594/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:32 2022] GET /identity/v3/role_assignments?user.id=a0f124ad0aeb44508cf8af1e3dc7bbb4&scope.project.id=dd3f755b7ca242658541b979d3a98edd&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:34.309088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead3fe1-bbad-4eb4-b34c-dbd776141643 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.309547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead3fe1-bbad-4eb4-b34c-dbd776141643 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.309663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead3fe1-bbad-4eb4-b34c-dbd776141643 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.311033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 568/1162] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:34.316273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.317419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.317419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.334632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:34.335799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:34.401015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 595/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:24:34.422612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead3fe1-bbad-4eb4-b34c-dbd776141643 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:34.464560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:34.525950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead3fe1-bbad-4eb4-b34c-dbd776141643 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:34.558181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:34.558821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.559106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.559381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.624066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:34.625521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:34.662433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:34.721278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 569/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:34.727410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:34.806539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:34.824362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437d3bc-86a7-4cbb-95ae-5bcf6f73434a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:34.857615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:34.858407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:34.858696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:34.858976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:34.866271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:34.867712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-336eb52f-ed7d-4de8-9bef-77e5df54df38 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:34.883099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:34.884769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:34.885937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 596/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:24:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:34.891007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:34.987462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8e352-6a2a-4c9c-9866-09d7e8142df6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:35.020658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:35.021331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:35.021617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:35.021893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:35.022611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:35.022916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:35.024260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fba1f-285e-4803-8b2b-3b19bc21879f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:35.031180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 570/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Jul 18 10:24:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:37.026095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e058-6a60-4511-8ef1-95d243fc6e5f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.026732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e058-6a60-4511-8ef1-95d243fc6e5f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.026732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e058-6a60-4511-8ef1-95d243fc6e5f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.028033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 597/1167] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:37.033096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382dc39-0e3b-43e9-9457-3b7725fb34a0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.033525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382dc39-0e3b-43e9-9457-3b7725fb34a0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.033640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382dc39-0e3b-43e9-9457-3b7725fb34a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.038655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:37.050371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2382dc39-0e3b-43e9-9457-3b7725fb34a0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:37.053728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2382dc39-0e3b-43e9-9457-3b7725fb34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:37.117313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 571/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:37.140029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e058-6a60-4511-8ef1-95d243fc6e5f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:37.238934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e058-6a60-4511-8ef1-95d243fc6e5f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:37.272327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fc746-84ee-4acb-a05f-19caa7386896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:37.273013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fc746-84ee-4acb-a05f-19caa7386896 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:37.273304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fc746-84ee-4acb-a05f-19caa7386896 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:37.273585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fc746-84ee-4acb-a05f-19caa7386896 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:37.274337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fc746-84ee-4acb-a05f-19caa7386896 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:37.275700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fc746-84ee-4acb-a05f-19caa7386896 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:37.337341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 598/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Mon Jul 18 10:24:37 2022] POST /identity/v3/domains => generated 267 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:39.366736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596a26e-af85-4b05-a43a-25571a1988f5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.367267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596a26e-af85-4b05-a43a-25571a1988f5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.367267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596a26e-af85-4b05-a43a-25571a1988f5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.368662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 572/1170] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:39.373990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.374402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.374545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.379567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.391137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:39.392323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:39.458168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 599/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:24:39.481845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596a26e-af85-4b05-a43a-25571a1988f5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.580965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596a26e-af85-4b05-a43a-25571a1988f5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.609482 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.614256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f638d-df87-4141-807e-3dcc36bf1e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.615099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f638d-df87-4141-807e-3dcc36bf1e72 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.615384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f638d-df87-4141-807e-3dcc36bf1e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.615659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f638d-df87-4141-807e-3dcc36bf1e72 None admin] PATH_INFO: `/v3/domains/94c94a71d96e4f9cb9c8732b3cf53e83` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.625767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f638d-df87-4141-807e-3dcc36bf1e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=94c94a71d96e4f9cb9c8732b3cf53e83)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.627201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f638d-df87-4141-807e-3dcc36bf1e72 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.629395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 573/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Mon Jul 18 10:24:39 2022] GET /identity/v3/domains/94c94a71d96e4f9cb9c8732b3cf53e83 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:39.635306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:39.731481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c1fee-da5f-4ad4-adc8-a2182dcbbeef None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:39.761375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:39.766207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f4aa3-90e0-420d-a6a4-d6771f717fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:39.767563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f4aa3-90e0-420d-a6a4-d6771f717fac None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:39.767563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f4aa3-90e0-420d-a6a4-d6771f717fac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:39.767563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f4aa3-90e0-420d-a6a4-d6771f717fac None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:39.768112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f4aa3-90e0-420d-a6a4-d6771f717fac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:39.770475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f4aa3-90e0-420d-a6a4-d6771f717fac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:39.867665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 600/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:24:39 2022] POST /identity/v3/projects => generated 361 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:41.892463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a78e3-2760-4461-a709-b669d1f6833f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:41.893066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a78e3-2760-4461-a709-b669d1f6833f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:41.893066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a78e3-2760-4461-a709-b669d1f6833f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:41.895122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 574/1174] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:41.900494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:41.900966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:41.901100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:41.919998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:41.921219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:41.984999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 601/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:42.010165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a78e3-2760-4461-a709-b669d1f6833f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:42.109274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a78e3-2760-4461-a709-b669d1f6833f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:42.141815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5eeb68-17eb-44ed-b73c-0ec562e16f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:42.142706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5eeb68-17eb-44ed-b73c-0ec562e16f9d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.142938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5eeb68-17eb-44ed-b73c-0ec562e16f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.143212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5eeb68-17eb-44ed-b73c-0ec562e16f9d None admin] PATH_INFO: `/v3/domains/94c94a71d96e4f9cb9c8732b3cf53e83` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.144429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5eeb68-17eb-44ed-b73c-0ec562e16f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=94c94a71d96e4f9cb9c8732b3cf53e83)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:42.146050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5eeb68-17eb-44ed-b73c-0ec562e16f9d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:42.147647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 575/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Mon Jul 18 10:24:42 2022] GET /identity/v3/domains/94c94a71d96e4f9cb9c8732b3cf53e83 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:42.153531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:42.248816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c499216-7431-4856-9547-a0d2ab37bfe7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:42.262582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:42.282995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cbfc8-5d92-426a-9abe-cca360a4c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:42.283678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cbfc8-5d92-426a-9abe-cca360a4c716 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:42.283961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cbfc8-5d92-426a-9abe-cca360a4c716 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:42.284240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cbfc8-5d92-426a-9abe-cca360a4c716 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:42.285602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cbfc8-5d92-426a-9abe-cca360a4c716 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:42.287035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cbfc8-5d92-426a-9abe-cca360a4c716 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:42.416160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 602/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:24:42 2022] POST /identity/v3/users => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:24:44.441825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e6956-b618-4842-8a84-97eb74afa24b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.442364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e6956-b618-4842-8a84-97eb74afa24b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.442364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e6956-b618-4842-8a84-97eb74afa24b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.443734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 576/1178] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:44.449143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.449542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.449718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.466994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:44.468194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:44.533695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 603/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:44.557913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e6956-b618-4842-8a84-97eb74afa24b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.622026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.662911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e6956-b618-4842-8a84-97eb74afa24b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.696022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.696843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.697132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.697410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.708075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:44.715781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:44.716954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:24:44.717856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 577/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:44.723011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.816898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.819655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c452db-91b6-47f9-af9e-6fb883e87c21 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.852870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.853527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:44.853818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:44.854102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:44.854851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:44.855165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:44.856524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:44.865071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 604/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:44.870740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:44.966532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098634-b038-434d-b306-9a30dbcfd43f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:44.985512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:44.999094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:44.999939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.000216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.000487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] PATH_INFO: `/v3/users/4306a806ba154727bce3ef17f859d2b5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.013088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] RBAC: Authorizing `identity:get_user(user_id=4306a806ba154727bce3ef17f859d2b5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:45.014506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:45.016545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 578/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:44 2022] GET /identity/v3/users/4306a806ba154727bce3ef17f859d2b5 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:45.021957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:45.120562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c12963-2dd0-4ff6-96e5-d1ad42d555e7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:45.153501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:45.154141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.154431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.154715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.184512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:45.221883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:45.224052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:45.321551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 605/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:45.327476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:45.425501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676fa47-b916-427f-a7bf-afb0ca896896 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:45.458891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d8bbf-6a09-475d-be8c-5e03748e0aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:45.459740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d8bbf-6a09-475d-be8c-5e03748e0aef None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.460024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d8bbf-6a09-475d-be8c-5e03748e0aef None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.460300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d8bbf-6a09-475d-be8c-5e03748e0aef None admin] PATH_INFO: `/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.461502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d8bbf-6a09-475d-be8c-5e03748e0aef None admin] RBAC: Authorizing `identity:get_project(project_id=fe3a0122a4354a679846f6ccdf2e4f1c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:45.462931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d8bbf-6a09-475d-be8c-5e03748e0aef None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:45.464629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 579/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:45 2022] GET /identity/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:45.470758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:45.530206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:45.568496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06304f4-865c-43a1-a15e-3dcaf3d00b52 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:45.601445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:45.602086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:45.602372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:45.602649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:45.603472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=fe3a0122a4354a679846f6ccdf2e4f1c, user.id=4306a806ba154727bce3ef17f859d2b5) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:45.603785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:45.605229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077e1eb-7efe-4532-b9af-487cf020fb49 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:45.610490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 606/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:45 2022] GET /identity/v3/role_assignments?user.id=4306a806ba154727bce3ef17f859d2b5&scope.project.id=fe3a0122a4354a679846f6ccdf2e4f1c&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:47.614434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e543b-3f1a-4e2b-88e6-bdd15d6789bb None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:47.614960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e543b-3f1a-4e2b-88e6-bdd15d6789bb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:47.614960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e543b-3f1a-4e2b-88e6-bdd15d6789bb None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:47.616276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 580/1186] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:47.621562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:47.621940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:47.622138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:47.639189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:47.640375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:47.705375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 607/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:47.728112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e543b-3f1a-4e2b-88e6-bdd15d6789bb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:47.774850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:47.827166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e543b-3f1a-4e2b-88e6-bdd15d6789bb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:47.860476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:47.861279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:47.861563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:47.861839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:47.871870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:47.879971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:47.881106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:24:47.886850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 581/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:47.887690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:47.943217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:47.985642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef71f64-6300-44fa-8d15-a0f880739401 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:48.017928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:48.018589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:48.018881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:48.019757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:48.020585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:48.020946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:48.022306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:48.030543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 608/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:48.036561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:48.182187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a6e1-8ba0-4da1-8e9f-dd9be2f47db7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:48.188849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:48.215485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:48.216344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:48.216629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:48.216937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] PATH_INFO: `/v3/users/4306a806ba154727bce3ef17f859d2b5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:48.218115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] RBAC: Authorizing `identity:get_user(user_id=4306a806ba154727bce3ef17f859d2b5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:48.219536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:48.221203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 582/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:48 2022] GET /identity/v3/users/4306a806ba154727bce3ef17f859d2b5 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:48.227796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:48.325098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294e7f-c83c-4e8d-b234-8d7e1b94ae04 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:48.342562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:48.359343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaafc36-1fb2-4cbf-a0c4-a15f12dd89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:48.360196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaafc36-1fb2-4cbf-a0c4-a15f12dd89aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:48.360488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaafc36-1fb2-4cbf-a0c4-a15f12dd89aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:48.360795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaafc36-1fb2-4cbf-a0c4-a15f12dd89aa None admin] PATH_INFO: `/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:48.361975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaafc36-1fb2-4cbf-a0c4-a15f12dd89aa None admin] RBAC: Authorizing `identity:get_project(project_id=fe3a0122a4354a679846f6ccdf2e4f1c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:48.363678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaafc36-1fb2-4cbf-a0c4-a15f12dd89aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:48.365386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 609/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:48 2022] GET /identity/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:48.371605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:48.474714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed66df-0a0e-43e5-a624-6f296f8d7c2e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:48.506549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50ce58-1bee-43e6-b1be-38076d5fd185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:48.507420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ce58-1bee-43e6-b1be-38076d5fd185 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:48.507708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ce58-1bee-43e6-b1be-38076d5fd185 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:48.507984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ce58-1bee-43e6-b1be-38076d5fd185 None admin] PATH_INFO: `/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c/users/4306a806ba154727bce3ef17f859d2b5/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:48.509976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50ce58-1bee-43e6-b1be-38076d5fd185 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3a0122a4354a679846f6ccdf2e4f1c, user_id=4306a806ba154727bce3ef17f859d2b5, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:48.511397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50ce58-1bee-43e6-b1be-38076d5fd185 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:48.579709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 583/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:24:48 2022] PUT /identity/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c/users/4306a806ba154727bce3ef17f859d2b5/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:50.557145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9521cb-a11c-47f8-8a25-d92c72092d64 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.557648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9521cb-a11c-47f8-8a25-d92c72092d64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.557648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9521cb-a11c-47f8-8a25-d92c72092d64 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.559002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 610/1193] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:50.564079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.564457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.564642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.581158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:50.583220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:50.609308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:50.650242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 584/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:24:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:24:50.674658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9521cb-a11c-47f8-8a25-d92c72092d64 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.774431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9521cb-a11c-47f8-8a25-d92c72092d64 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.806922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.807753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.807990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.808269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.818595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:24:50.826151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:24:50.827237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:24:50.828300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 611/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:24:50.833598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:50.931461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d6c0-13bb-4d67-a477-c62efcad7bc4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:50.964178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:50.964855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:50.965140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:50.965417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:50.966156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:50.966464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:50.967812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:50.976180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 585/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:50.981935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.027609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.080055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e3c03-1500-4008-a732-d6fd5362fecd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.114199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.115057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.115344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.115623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] PATH_INFO: `/v3/users/4306a806ba154727bce3ef17f859d2b5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.116848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] RBAC: Authorizing `identity:get_user(user_id=4306a806ba154727bce3ef17f859d2b5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.118283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.119879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 612/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Jul 18 10:24:50 2022] GET /identity/v3/users/4306a806ba154727bce3ef17f859d2b5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:51.125559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.152458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.222693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb53c-c0d9-4068-8c84-7576bc9a06f4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.254962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.255605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.255892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.256168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.321800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.323236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.354806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.419316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 586/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:24:51.425425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.521548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e080599-80b2-45f3-9946-83405164ea2b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.535145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.554797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb21926-b64c-49cd-ad9a-87f6b66eff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.555660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb21926-b64c-49cd-ad9a-87f6b66eff27 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.555947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb21926-b64c-49cd-ad9a-87f6b66eff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.556227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb21926-b64c-49cd-ad9a-87f6b66eff27 None admin] PATH_INFO: `/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.557417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb21926-b64c-49cd-ad9a-87f6b66eff27 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3a0122a4354a679846f6ccdf2e4f1c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.558848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb21926-b64c-49cd-ad9a-87f6b66eff27 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.560533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 613/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/projects/fe3a0122a4354a679846f6ccdf2e4f1c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:51.567203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:51.663188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcab63a-3172-420c-bea3-00fafe014189 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:51.688233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:51.697187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:51.701448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:51.701448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:51.701448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:51.701448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=fe3a0122a4354a679846f6ccdf2e4f1c, user.id=4306a806ba154727bce3ef17f859d2b5) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:51.701448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:51.701448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182de0d-8156-4ed8-86cd-ae06d3ddb84f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:51.707324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 587/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:24:51 2022] GET /identity/v3/role_assignments?user.id=4306a806ba154727bce3ef17f859d2b5&scope.project.id=fe3a0122a4354a679846f6ccdf2e4f1c&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:25:01.568847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=9 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:01.719108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:04.828917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688453-e5f1-42b4-9ef0-7a2ee746e995 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.829504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688453-e5f1-42b4-9ef0-7a2ee746e995 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.829504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688453-e5f1-42b4-9ef0-7a2ee746e995 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.830844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 614/1201] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:04.836533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:04.837023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:04.837124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:04.854181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:04.855353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:04.920931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 588/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:04.945544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68688453-e5f1-42b4-9ef0-7a2ee746e995 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:05.047803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68688453-e5f1-42b4-9ef0-7a2ee746e995 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:05.082789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:05.083507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:05.083790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:05.084062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:05.094024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:05.095460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:05.112814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:05.114030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:05.115014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:05.115014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:05.115014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:05.115014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:05.115014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 615/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:05.120246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:05.224609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ee75a-17ea-4aea-b9a7-65c6864aca57 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:05.252937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:05.258646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:05.259297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:05.259578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:05.259852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:05.260571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:05.260906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:05.262280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ae0ec-ae1b-4c1c-8ed2-9188886cd8aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:05.273643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 589/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:05.279478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:05.331048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:05.376649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7afc4-67d3-43d6-bf0e-313f4b0e38eb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:05.408690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd37027-3386-49e3-80bb-b48e5ea6b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:05.409372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37027-3386-49e3-80bb-b48e5ea6b051 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:05.409658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37027-3386-49e3-80bb-b48e5ea6b051 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:05.409935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37027-3386-49e3-80bb-b48e5ea6b051 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:05.410650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd37027-3386-49e3-80bb-b48e5ea6b051 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:05.412058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd37027-3386-49e3-80bb-b48e5ea6b051 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:05.505739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 616/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Jul 18 10:26:05 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:07.580230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0f25-dbdb-40ba-a72b-eabd5f9433d6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.580812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0f25-dbdb-40ba-a72b-eabd5f9433d6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.580812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0f25-dbdb-40ba-a72b-eabd5f9433d6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.582442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 590/1206] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:07.587824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.588287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.588407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.605262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:07.606456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:07.672165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 617/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:26:07.695568 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0f25-dbdb-40ba-a72b-eabd5f9433d6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.794705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0f25-dbdb-40ba-a72b-eabd5f9433d6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.827710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.828640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.828964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.829244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.840075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:07.847914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:07.849159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:07.850258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 591/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:07.855346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:07.898141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:07.956103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3174b8-3bc1-4018-8dd0-a9f67e57b3a3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:07.989057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:07.989723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:07.990068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:07.990305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:07.991351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:07.991834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:07.993787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.002476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 618/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:26:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:08.008428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.057598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:08.106074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a825e-4200-4960-b8cc-33e191fff820 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.138862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.139644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.139930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.140211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.150554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.151981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:08.169370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:08.170717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:08.172020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:08.172020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:08.172020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:08.172020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:08.172020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 592/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:08.178674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.280375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:08.283779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c81c6-7cb6-42bb-b399-cbfaf208707e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.318066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.319149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.319666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.320172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.321454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:08.330319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.330319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.337713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 619/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:08.343358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.444769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53294e3e-908b-47b7-9dee-3ac59c84d1ff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.446182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:08.478366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.479194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.479526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.479804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.487370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.488813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:08.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:08.505761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:08.506759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 593/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:08.511638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.612563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ff0fb-58e2-446c-8796-390ceabebea5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.645432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.646101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.646386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.646666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.647496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:08.647808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.649264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.666450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 620/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:08.672585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.769893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a096ee0-aa24-4418-82c6-7e9ac8154a64 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:08.803685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:08.804332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:08.804614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:08.813719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:08.835367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:08.870831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:08.872298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:08.973908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 594/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:08.979711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:08.998620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:09.079391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330189f-0fe3-4970-ab57-7dc20c818c95 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.112965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.113768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.114060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.114342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.124152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.126168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:09.144280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:09.148775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:09.149985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:09.149985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:09.149985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:09.149985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:09.149985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 621/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:09.151381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:09.213414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:09.252457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd5245-2b21-48a8-997a-b8da1e1e09d7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.286046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.286707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.287004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.287278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.288002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:09.288313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.289713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.301101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 595/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:09.306803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:09.378086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:09.407351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e04b6-d3fb-428b-9c51-b74f173e722d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.440700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.441795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.442554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.442842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.449443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.450889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:09.465591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:09.466661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:09.467852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 622/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:09.472871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:09.578975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3594c-2411-4547-8174-6e00f7e3902b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.602171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:09.614139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.614796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.615084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.615370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.616157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:09.616470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.618025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ff0eb-a0ad-4ca6-9329-38d6b76b2dc5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.629874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 596/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:09.636197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:09.738584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c2d63-bb12-43a5-93ea-7d257635087b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:09.766172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:09.772225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:09.773068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:09.773359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:09.773643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:09.774470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=f7b860ca751f4eb7bcc72617e05abd30) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:09.774789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:09.776380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafd53a-eeb4-4cfd-a06c-dbe0c7f926ca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:09.782497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 623/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:09 2022] GET /identity/v3/role_assignments?user.id=f7b860ca751f4eb7bcc72617e05abd30&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:11.787272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9d923-515b-42be-819f-51c756bff465 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:11.787804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9d923-515b-42be-819f-51c756bff465 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:11.787921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9d923-515b-42be-819f-51c756bff465 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:11.789266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 597/1220] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:11.794334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:11.794770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:11.794909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:11.811920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:11.813155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:11.880280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 624/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:11.902933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9d923-515b-42be-819f-51c756bff465 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.009404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9d923-515b-42be-819f-51c756bff465 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.042832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.043551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.043834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.044107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.054877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:12.062698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:12.069881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:12.070981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 598/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:26:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:12.071858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.167509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45822645-73d2-4f59-977e-fc0bb8b9fc93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.190932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:12.201215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.201876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.202165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.202445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.203690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:12.204000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.205389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.213525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 625/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:12.219859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.263238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:12.320022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba840b7d-bde7-4296-975c-1e7713318c58 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.353732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.354519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.354846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.355137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.365720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.367162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:12.383796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:12.384843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:12.385780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:12.385780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:12.385780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:12.385780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:12.385780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 599/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:12.391399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.489422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a8d82-809a-4c2e-915e-16272b815bf4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.527879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.528543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.528872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.529154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.529881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:12.530196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.532185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.543637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 626/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:12.549476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.640518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:12.646887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173021-d943-4b08-bbba-191ad7544337 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.680410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.681244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.681532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.681812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.689354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.695038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:12.704960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:12.705947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:12.707416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 600/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:12.712247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.759286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:12.811239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67ba7-71a5-47d8-ae75-cb98b0d97bb9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:12.844212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:12.844899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:12.845188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:12.845468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:12.846298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:12.846612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:12.848052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:12.866112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 627/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:12.872077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:12.971071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c50bd-2b93-4fcd-84d5-a62cae603f01 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.003249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.004023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.004307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.004583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.016015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.017470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:13.034940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:13.040458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:13.040458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:13.040458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:13.040458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:13.040458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 601/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:13.042003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.139099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e534164-1902-4ad5-8649-0ef624768f6d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.141909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.173104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.173765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.174055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.174339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.175069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:13.175383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.176767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.187644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 628/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:13.193606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.203403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.294196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ece1c-2d10-4169-98cd-62679d0a0da5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.327357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.328786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.329075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.329354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.337591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.339118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:13.353716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:13.354766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:13.355972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 602/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:13.361185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.461670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c01fe-1a7c-4e83-90c2-2c62ae2b5e19 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.494765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.495448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.495742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.496026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.496843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:13.499059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.499059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc52678-1a08-4bfe-aba9-2f508a41adac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.511276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 629/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:13.518501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:13.586742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:13.619685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b658e4d-86fc-43a8-b991-976a97d8b14d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:13.657882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909490d8-bf84-4808-90bd-8db8df3a929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:13.658773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909490d8-bf84-4808-90bd-8db8df3a929e None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:13.659057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909490d8-bf84-4808-90bd-8db8df3a929e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:13.659335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909490d8-bf84-4808-90bd-8db8df3a929e None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/f7b860ca751f4eb7bcc72617e05abd30/roles/ee4e96dd391c4ca482a95821cccd65cb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:13.677497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909490d8-bf84-4808-90bd-8db8df3a929e None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=f7b860ca751f4eb7bcc72617e05abd30, role_id=ee4e96dd391c4ca482a95821cccd65cb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:13.678931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909490d8-bf84-4808-90bd-8db8df3a929e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:13.797650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 603/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:13 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/f7b860ca751f4eb7bcc72617e05abd30/roles/ee4e96dd391c4ca482a95821cccd65cb => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:26:15.775043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e2d4-31a7-4f70-8266-d2abf82b04c9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.775579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e2d4-31a7-4f70-8266-d2abf82b04c9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.775681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e2d4-31a7-4f70-8266-d2abf82b04c9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.777024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 630/1233] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:15.782273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:15.782701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:15.782840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:15.798810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:15.800015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:15.865716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 604/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:15.889404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e2d4-31a7-4f70-8266-d2abf82b04c9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:15.914898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:15.991069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e2d4-31a7-4f70-8266-d2abf82b04c9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.023904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.024765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.025004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.025332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.038827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:16.047052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:16.048037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:16.048978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 631/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Jul 18 10:26:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:16.054842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.100674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.151649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcade2-9e21-43d1-ba45-419d9acf840e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.183899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.184555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.184875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.185162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.185909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:16.186219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.187776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.195763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 605/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:16.201677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.289847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.305061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ac17-6f02-4cfa-89fa-9a76b36d168a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.339494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.340296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.340582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.340887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.351153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.352598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:16.372250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:16.377568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:16.378688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:16.378688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:16.378688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:16.378688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:16.378688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 632/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:16.379899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.478859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17331b-c3b0-4382-ad9c-cba755c4ba3f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.488202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.512231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.512912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.513198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.513475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.514194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:16.514502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.516639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.527447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 606/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:16.533489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.632864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0649c9e-c8bb-4325-9e25-d43e4e5e62ca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.672234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.673193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.673482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.673762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.676532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:16.681127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.682572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:16.697578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:16.698724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:16.703870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 633/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:26:16.704753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.802836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7abd8-40ea-4e07-92ab-85e090cd83f5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.836650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.837335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.837620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.837897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:16.838725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:16.839038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:16.840451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:16.857166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 607/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:16.863586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:16.961866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaacda7-c933-4b94-b404-379fe9627364 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:16.995473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:16.996125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:16.996416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:16.996717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:17.056671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:17.063791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:17.065280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:17.162726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 634/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:17.168779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:17.196121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:17.275594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df693ffa-c417-4fc2-b612-4880be171c32 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:17.301080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:17.301865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:17.302147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:17.302423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:17.312165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:17.313622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:17.330577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:17.331894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:17.332872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:17.332872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:17.332872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:17.332872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:17.332872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 608/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:17.338023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:17.433046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:17.438839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5d67-b8c3-429e-84e3-cf2e3b7260e8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:17.472892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:17.473548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:17.473835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:17.474113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:17.474834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:17.475144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:17.476503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:17.487424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 635/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:17.493266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:17.573050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:17.596537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dfbad-c579-4aff-a289-cb3864b23027 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:17.630951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:17.631727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:17.632025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:17.632296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:17.638721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:17.641490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:17.655309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:17.656953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:17.657944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:17.658414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 609/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:17.663535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:17.767081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1990cc-ff80-4418-a816-84ac96a49d19 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:17.800959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:17.802912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:17.802912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:17.802912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:17.802912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:17.803228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:17.804617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ce94-94eb-4ec3-8cea-c2324a93dc76 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:17.816971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 636/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:17.823182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:17.925772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822bb10-46c5-4c5d-be14-660c701fa93d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:17.960183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:17.960856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:17.961139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:17.961412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:17.962259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] RBAC: Adding query filter params (role.id=ee4e96dd391c4ca482a95821cccd65cb, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=f7b860ca751f4eb7bcc72617e05abd30) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:17.962637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:17.969235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc5578-17ea-48af-9bb5-79122327f650 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:17.970208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 610/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:17 2022] GET /identity/v3/role_assignments?user.id=f7b860ca751f4eb7bcc72617e05abd30&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=ee4e96dd391c4ca482a95821cccd65cb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:20.069639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53707e0-52b0-4e4c-808f-621823135b59 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.070206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53707e0-52b0-4e4c-808f-621823135b59 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.070312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53707e0-52b0-4e4c-808f-621823135b59 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.071785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 637/1247] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:20.077263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.078277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.078439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.095173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:20.095913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:20.097148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:20.162047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 611/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:20.185474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53707e0-52b0-4e4c-808f-621823135b59 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:20.203134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:20.286887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53707e0-52b0-4e4c-808f-621823135b59 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.320146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.320899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.321203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.321477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.331546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:20.339448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:20.340591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:20.341687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 638/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:20.347932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:20.451670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6cc87-b8e5-41bd-9c2c-74e11aaacfe4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.482593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:20.487396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.488056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.488338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.488615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.489408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:20.489716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.491066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:20.498884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 612/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:20.505025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:20.585366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:20.607607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f32a1-26fa-4815-bbc4-51054c1dab18 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.641257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.642032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.642316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.642592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.652514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.654070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:20.672591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:20.673749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:20.674650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:20.674650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:20.674650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:20.674650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:20.674650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 639/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:20.679526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:20.785699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3287b38-0d2a-460d-81f0-8deb0db01053 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.828766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.833836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.833836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.833836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.833836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:20.833836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.833836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:20.845531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 613/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:20.851352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:20.951681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b2410-6b8c-4cde-b570-1b8eae39b315 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:20.979112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:20.986780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:20.987581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:20.987868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:20.988142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:20.995013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:20.996816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:21.011368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:21.012393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:21.013350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 640/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:21.019405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.080208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.123895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9b1bf-9297-4c60-9d9d-e3f2a439605d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.157681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.158343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.158625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.158899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.159725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:21.160034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.161581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.178611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 614/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:21.189028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.283888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb2cfa-8853-4ae4-8387-12265af11b7a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.318335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.318981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.319267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.319544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.373027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.385509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.386895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.481716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 641/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:21.488290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.585497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fdb8c-5a6f-4e1f-a368-6838d5732c7b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.599006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.619465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.620246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.620528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.620902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.630705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.632144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:21.649854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:21.650931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:21.657201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:21.657201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:21.657201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:21.657201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:21.657201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 615/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:21.657534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.742171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:21.754270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b61ca6-4807-4af1-85a0-a19ca0eb0b69 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.788251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.788945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.789231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.789511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.790229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:21.790536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.791898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.803464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 642/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:21.809223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:21.908819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9afa-e21a-44c6-9a1a-75adc60091ba None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:21.944423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:21.945220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:21.945503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:21.945778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:21.952898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:21.954466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:21.969133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:21.973407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:21.974500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 616/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:21.976257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.075578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0a7f-7f86-48a8-8830-5919125e3009 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.109729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.110366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.110659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.110946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.111729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:22.112047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:22.113509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25263187-fea5-4f96-babf-a55db785d22d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:22.125297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 643/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:22.131599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:22.164393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:22.236331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d23a5-f032-4c17-a472-746d25d6cb9d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:22.269892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:22.270546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:22.270830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:22.271106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:22.271935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=f7b860ca751f4eb7bcc72617e05abd30) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:22.272249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:22.273696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9588f6-56b5-47c6-b647-79dc16b0fd7f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:22.279127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 617/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:22 2022] GET /identity/v3/role_assignments?user.id=f7b860ca751f4eb7bcc72617e05abd30&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=e4cb7641166847b497698b97084ca96b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:24.254246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28c576-0fe4-43eb-8472-22fb1d8c7289 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.254768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28c576-0fe4-43eb-8472-22fb1d8c7289 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.254860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28c576-0fe4-43eb-8472-22fb1d8c7289 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.256167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 644/1261] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:24.261425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.261833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.261980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.279172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:24.281094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:24.346550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 618/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:24.369280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28c576-0fe4-43eb-8472-22fb1d8c7289 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.380564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.469307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28c576-0fe4-43eb-8472-22fb1d8c7289 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.503418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.509792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.509792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.509792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.514561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:24.522577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:24.523681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:24.524750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 645/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:24.529768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.548917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.626272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de559f21-9312-425a-9a3e-374a1581f242 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.658582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.659239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.659523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.659796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.660543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:24.660882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.662234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.669938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 619/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:24.675721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.742004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.775236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c9247-8869-4e6e-9f47-4696ed5f55a9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.808730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.809513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.809803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.810129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.820385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.822294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:24.838810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:24.839881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:24.841057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:24.841057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:24.841057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:24.841057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:24.841057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 646/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:24.846416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:24.926648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:24.945371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3ab56-daa5-4785-8d71-fc18e5d20c46 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:24.982481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:24.983142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:24.983514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:24.983860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:24.984590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:24.984929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:24.986287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:24.997912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 620/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:25.003873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.103394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d61736-ca09-4e3d-861f-ccaa28a8eee5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.122301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:25.136159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.136996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.137289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.137608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.145480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.146978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:25.161824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:25.162944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:25.164112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 647/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:25.168971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.269205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3917c-4af2-4476-820f-542cd7bbaf21 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.302418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.303072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.303359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.303634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.304457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:25.304793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.306247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.315490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:25.323725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 621/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Jul 18 10:26:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:25.330242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.430406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71ec9-665f-4167-ac1c-a1e5d666c831 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.466298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.467338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.467683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.468203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.480188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.481668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:25.498549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:25.500200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:25.501418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:25.501418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:25.501418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:25.501418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:25.501418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 648/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:25.506220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.607516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82bcaa-8aba-4df0-8a0e-ca911c46f1fc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.643007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.643674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.643960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.644238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.645002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:25.645313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.646777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.658173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 622/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:25.664019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.695779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:25.762681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31080229-3d21-4ac1-bcaa-95f7c9cace4c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.797173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.797966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.798254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.798537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.805214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.806672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:25.822882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:25.824713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:25.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 649/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:25.830613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:25.889069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:25.931648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928698a2-cfc6-40a2-8904-1868318b9f73 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:25.968636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:25.969443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:25.969732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:25.970009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:25.970790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:25.971111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:25.972576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f6da1-dc12-4fa7-a734-59e908636c5a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:25.985300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 623/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:25.991513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:26.086874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:26.095233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af656b6c-2995-4fe8-8d1c-6dd61cc7e977 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:26.127993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ff4d6-f4e7-4ae6-a918-250610315244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:26.128902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ff4d6-f4e7-4ae6-a918-250610315244 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:26.129252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ff4d6-f4e7-4ae6-a918-250610315244 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:26.129531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ff4d6-f4e7-4ae6-a918-250610315244 None admin] PATH_INFO: `/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/f7b860ca751f4eb7bcc72617e05abd30/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:26.131622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ff4d6-f4e7-4ae6-a918-250610315244 None admin] RBAC: Authorizing `identity:create_grant(project_id=12759b13d97d46d692472e8cd8550ab2, user_id=f7b860ca751f4eb7bcc72617e05abd30, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:26.133060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ff4d6-f4e7-4ae6-a918-250610315244 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:26.282889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 650/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Jul 18 10:26:25 2022] PUT /identity/v3/projects/12759b13d97d46d692472e8cd8550ab2/users/f7b860ca751f4eb7bcc72617e05abd30/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:26:28.288990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c86e0-9a36-4998-ab95-602de42ada96 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.289480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c86e0-9a36-4998-ab95-602de42ada96 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.289596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c86e0-9a36-4998-ab95-602de42ada96 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.290961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 624/1274] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:28.296113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.296537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.296704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.313897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:28.315093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:28.380902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 651/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:28.404596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c86e0-9a36-4998-ab95-602de42ada96 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:28.503339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c86e0-9a36-4998-ab95-602de42ada96 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:28.520994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:28.537362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:28.538153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.538438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.538718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.548711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:26:28.556302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:28.558067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:28.559357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 625/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Jul 18 10:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:28.564253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:28.644328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:28.661683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651e031-027e-4f7c-883b-21ddd9718f73 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:28.695661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:28.696320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.696609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.696918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.697670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:28.697984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:28.699343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:28.707778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 652/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Jul 18 10:26:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:28.713667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:28.812723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ec846-76fc-4a52-855b-02f2acbb3652 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:28.845583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:28.846365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:28.846649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:28.846927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:28.857300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:28.859580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:28.877062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:28.877938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:28.878851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:28.878851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:28.878851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:28.878851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:28.878851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 626/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:28.884202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:28.985474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277a040-fba4-4ae3-8c66-2e7abfa5f8fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:29.020528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:29.021205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:29.021490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:29.021764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:29.022535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:29.022845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:29.025075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:29.037284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 653/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:29.042947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:29.073558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:29.140274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31367-1f02-46eb-a670-c0a7c4ea56b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:29.174123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:29.174921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:29.175208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:29.175487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:29.183763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:29.185225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:26:29.199795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:29.203045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:29.204250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 627/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:29.207806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:29.211642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:29.306643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf45eb-2bde-4aaa-b98d-3cb4eb3da129 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:29.340365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d262488-67f7-431a-b441-2541101da99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:29.340998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d262488-67f7-431a-b441-2541101da99d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:29.341281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d262488-67f7-431a-b441-2541101da99d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:29.341556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d262488-67f7-431a-b441-2541101da99d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:29.342383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d262488-67f7-431a-b441-2541101da99d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:29.342744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d262488-67f7-431a-b441-2541101da99d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:29.344164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d262488-67f7-431a-b441-2541101da99d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:29.360880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 654/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Jul 18 10:26:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:26:29.366499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:29.457512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:29.466670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2f16-cf0b-45cf-8d90-aad98ca289e1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:29.499201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:29.499837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:29.500123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:29.500399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:29.566366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:29.567809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:29.654241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:29.664400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 628/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Jul 18 10:26:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:26:29.670318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d262488-67f7-431a-b441-2541101da99d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:29.731560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:29.774278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d262488-67f7-431a-b441-2541101da99d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:29.807328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:29.808106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:29.808384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:29.808655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:29.818827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:29.820270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:26:29.837138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:29.838207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:29.839137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:26:29.839137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:26:29.839137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:26:29.839137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:29.839137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 655/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:29.844831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:29.946945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df311-cf2a-44bd-965b-e57fd3ce12c2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:29.979941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:29.980591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:29.980911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:29.981193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:29.981917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:29.982227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:29.989487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:29.994574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 629/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Jul 18 10:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:30.001082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:30.101402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d805693-92f6-4cbe-80cd-948f6d8379e6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:30.114940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:30.135702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:30.136492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.136811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.137098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.144199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:30.145676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:26:30.161367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:30.169858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 18 10:26:30.171528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 656/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Jul 18 10:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:30.172024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:30.222100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:30.271979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc571492-934d-4282-8d6e-1361c425a962 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:30.304969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:30.305621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.305902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.306178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.306958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:30.307266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:30.308704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362cf2e-4af1-4f75-8413-5881bd600128 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:30.320585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 630/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Jul 18 10:26:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:30.327622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:30.425626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2998e-2d82-4f63-935c-398eabe867a6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:30.459637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:30.460288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:30.460573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:30.460900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:30.461731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] RBAC: Adding query filter params (role.id=e4cb7641166847b497698b97084ca96b, scope.project.id=12759b13d97d46d692472e8cd8550ab2, user.id=f7b860ca751f4eb7bcc72617e05abd30) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:30.462044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:30.463537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed17447-666f-46c4-a1fd-1ce8c10fdb8a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:30.469962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 657/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Jul 18 10:26:30 2022] GET /identity/v3/role_assignments?user.id=f7b860ca751f4eb7bcc72617e05abd30&scope.project.id=12759b13d97d46d692472e8cd8550ab2&role.id=e4cb7641166847b497698b97084ca96b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:32.534463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ebe2-8101-44b5-997e-590ae22f99e0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:32.535011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ebe2-8101-44b5-997e-590ae22f99e0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:32.535011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ebe2-8101-44b5-997e-590ae22f99e0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:32.536314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 631/1288] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:32.541733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:32.542126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:32.542323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:32.559501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:32.560839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:32.591132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:32.627805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 658/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:32.649428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6ebe2-8101-44b5-997e-590ae22f99e0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:32.753485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6ebe2-8101-44b5-997e-590ae22f99e0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:32.764657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:32.787234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:32.788004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:32.788325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:32.788603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:32.789361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:32.790752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:32.803346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:32.809436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:32.810362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 632/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:26:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:32.811333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:32.910590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be2fb7-dd04-4aed-abc2-6ca72caedf10 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:32.943885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:32.944537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:32.944855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:32.945126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:32.945826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:32.946132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:32.947482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6f7af-3fff-4a9f-ac3e-72acbd426457 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:32.952904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 659/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Jul 18 10:26:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:32.958993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:33.058081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d3559-4bef-474a-be06-29a81d03eb98 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:33.092431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:33.093314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:33.093603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:33.093879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:33.094585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:33.094897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:33.096251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3a45b-61f8-4dce-b33a-ec47de1e9cf8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:33.100697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 633/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Jul 18 10:26:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:35.135312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107a21c-a38d-4859-b10f-c32de195df18 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:35.135850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107a21c-a38d-4859-b10f-c32de195df18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:35.135942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107a21c-a38d-4859-b10f-c32de195df18 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:35.137264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 660/1293] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:35.142370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96742f41-d2b8-40fb-a5e3-f280ca3c617b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:35.142769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96742f41-d2b8-40fb-a5e3-f280ca3c617b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:35.142929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96742f41-d2b8-40fb-a5e3-f280ca3c617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:35.160006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-96742f41-d2b8-40fb-a5e3-f280ca3c617b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:35.161213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-96742f41-d2b8-40fb-a5e3-f280ca3c617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:35.191862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:35.230955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 634/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:35.254719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107a21c-a38d-4859-b10f-c32de195df18 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:35.286388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:35.354768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107a21c-a38d-4859-b10f-c32de195df18 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:35.389278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e75cd5-0cb0-4bd9-b9b9-d78983f35db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:35.390079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e75cd5-0cb0-4bd9-b9b9-d78983f35db7 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:35.390371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e75cd5-0cb0-4bd9-b9b9-d78983f35db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:35.390653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e75cd5-0cb0-4bd9-b9b9-d78983f35db7 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:35.391381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e75cd5-0cb0-4bd9-b9b9-d78983f35db7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:35.392774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e75cd5-0cb0-4bd9-b9b9-d78983f35db7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:35.486757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 661/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Jul 18 10:26:35 2022] POST /identity/v3/services => generated 246 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:37.515125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e6f06-d48d-419f-a5f3-15f6d76fbc3a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:37.515683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e6f06-d48d-419f-a5f3-15f6d76fbc3a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:37.515683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e6f06-d48d-419f-a5f3-15f6d76fbc3a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:37.517142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 635/1296] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:37.522570 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:37.523052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:37.523144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:37.540904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:37.542145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:37.625787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 662/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:37.650469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e6f06-d48d-419f-a5f3-15f6d76fbc3a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:37.760455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e6f06-d48d-419f-a5f3-15f6d76fbc3a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:37.801158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:37.801882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:37.802165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:37.802442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:37.803099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:37.804468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:37.816878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:37.818233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:37.819250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 636/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:26:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:37.824482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:37.843278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:37.923592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028cad7-d69c-4877-9c97-41af738400c9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:37.958520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:37.959179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:37.959580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:37.959863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:37.960571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:37.960921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:37.962278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:37.968463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 663/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Jul 18 10:26:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:37.974076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:38.004957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:38.075445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5389d29-c7fc-43bb-847b-d7a7b2d3b3ff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:38.108803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:38.109452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:38.109733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:38.110008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:38.110791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] RBAC: Adding query filter params (interface=public, service_id=2ae4e5dfc35c41e6a682e52c2afcdf2d, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:38.111101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:38.112446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eaf74-35f7-4942-951c-b90cdf1c6329 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:38.117443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 637/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Jul 18 10:26:37 2022] GET /identity/v3/endpoints?service_id=2ae4e5dfc35c41e6a682e52c2afcdf2d&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:38.123250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:38.204627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:38.222536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdcf7d-7e6d-43bb-9ee2-3081643b9ef0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:38.255582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c07a98-bd77-4751-ae0a-827befa844b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:38.256247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07a98-bd77-4751-ae0a-827befa844b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:38.256536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07a98-bd77-4751-ae0a-827befa844b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:38.256848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07a98-bd77-4751-ae0a-827befa844b7 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:38.257536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c07a98-bd77-4751-ae0a-827befa844b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:38.258949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c07a98-bd77-4751-ae0a-827befa844b7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:38.266822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 664/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Jul 18 10:26:38 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:26:40.262864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c115f33-d3b9-4752-badf-5cb755325efe None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:40.263389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c115f33-d3b9-4752-badf-5cb755325efe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:40.263477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c115f33-d3b9-4752-badf-5cb755325efe None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:40.265759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 638/1302] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:40.271191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:40.271580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:40.271761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:40.289069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:40.290265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:40.356468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 665/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:26:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:26:40.379133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c115f33-d3b9-4752-badf-5cb755325efe None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:40.479199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:40.485849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c115f33-d3b9-4752-badf-5cb755325efe None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:40.518672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:40.519444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:40.519731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:40.520009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:40.520658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:40.522041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:26:40.533846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:26:40.535010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jul 18 10:26:40.536086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 639/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Jul 18 10:26:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:26:40.541160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:40.616371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:40.638328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fd36-dda0-43b3-8803-d60bb35ef29a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:40.671744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:40.672401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:40.672709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:40.672990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:40.673693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:26:40.673999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:40.675415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b9d5-c5ae-483a-9ca1-a5fa60c74154 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:40.681328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 666/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Jul 18 10:26:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:40.687905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:40.786197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922b43b-1c54-4361-a442-af66d5f107b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:40.819430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac289c-500f-4011-affa-4bb97825d6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:40.820070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac289c-500f-4011-affa-4bb97825d6ae None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:40.820351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac289c-500f-4011-affa-4bb97825d6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:40.820639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac289c-500f-4011-affa-4bb97825d6ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:40.821389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac289c-500f-4011-affa-4bb97825d6ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:40.822740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac289c-500f-4011-affa-4bb97825d6ae None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:40.904995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 640/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Jul 18 10:26:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:26:50.678279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=30 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:50.900143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:19.476562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13a3d-fa68-42c4-b9ce-17a5d74d6023 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:19.477549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13a3d-fa68-42c4-b9ce-17a5d74d6023 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:19.478041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13a3d-fa68-42c4-b9ce-17a5d74d6023 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:19.479924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 667/1307] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:19.493248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc07dff-dbfa-4a01-96d1-7b8efa81a12c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:19.493248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc07dff-dbfa-4a01-96d1-7b8efa81a12c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:19.493248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc07dff-dbfa-4a01-96d1-7b8efa81a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:19.581335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-acc07dff-dbfa-4a01-96d1-7b8efa81a12c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:19.582654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-acc07dff-dbfa-4a01-96d1-7b8efa81a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:19.705144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 641/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:19.761585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62170021-0928-4f21-ac63-2477a7958515 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:19.763303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62170021-0928-4f21-ac63-2477a7958515 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:19.764302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62170021-0928-4f21-ac63-2477a7958515 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:19.765927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 668/1309] 10.222.0.47 () {54 vars in 965 bytes} [Mon Jul 18 10:34:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:19.772961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027051ba-9a6c-4bdf-8759-1de579b5b398 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:19.773380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027051ba-9a6c-4bdf-8759-1de579b5b398 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:19.773761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027051ba-9a6c-4bdf-8759-1de579b5b398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:19.862650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-027051ba-9a6c-4bdf-8759-1de579b5b398 None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:19.864266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-027051ba-9a6c-4bdf-8759-1de579b5b398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:19.886829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:19.951744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 642/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Mon Jul 18 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:19.962031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62170021-0928-4f21-ac63-2477a7958515 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:19.964060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-62170021-0928-4f21-ac63-2477a7958515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:20.088766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62170021-0928-4f21-ac63-2477a7958515 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:20.165842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:20.166578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:20.166975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:20.167363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:20.169160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:20.226839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:20.292558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:20.294242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c76fd3-125f-40fc-a3e1-90afde4aa38e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:20.315705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 669/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:22.748277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea370d-ba0d-4d2c-83a2-a2f1942cac00 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:22.748848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea370d-ba0d-4d2c-83a2-a2f1942cac00 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:22.748848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea370d-ba0d-4d2c-83a2-a2f1942cac00 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:22.750453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 643/1312] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:22.755664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c9e4-ee75-4dc7-831b-3a23b4b67379 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:22.756121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c9e4-ee75-4dc7-831b-3a23b4b67379 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:22.756223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c9e4-ee75-4dc7-831b-3a23b4b67379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:22.844659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9f83c9e4-ee75-4dc7-831b-3a23b4b67379 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:22.846638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f83c9e4-ee75-4dc7-831b-3a23b4b67379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:22.970673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 670/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:23.010966 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea370d-ba0d-4d2c-83a2-a2f1942cac00 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:23.012584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f4ea370d-ba0d-4d2c-83a2-a2f1942cac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:23.126413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea370d-ba0d-4d2c-83a2-a2f1942cac00 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:23.200129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:23.202754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:23.203397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:23.203686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:23.203967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:23.205646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:23.317554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:23.318993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1990ad-e4a9-401e-aded-0865dd669a22 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:23.337580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 644/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:26.001317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb49d6f-4fd3-4f0a-81e8-3d563a0f0da6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:26.001843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb49d6f-4fd3-4f0a-81e8-3d563a0f0da6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:26.001843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb49d6f-4fd3-4f0a-81e8-3d563a0f0da6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:26.003172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 671/1315] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:26.008458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ce20-739a-45d3-853c-b54742791dca None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:26.009002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ce20-739a-45d3-853c-b54742791dca None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:26.009110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ce20-739a-45d3-853c-b54742791dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:26.072576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-56e2ce20-739a-45d3-853c-b54742791dca None None] MFA Rules not processed for user `4b3ba696b60e4dc9b69780cc894e58a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:26.073818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-56e2ce20-739a-45d3-853c-b54742791dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:26.125320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:26.169312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 645/1316] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:26.209658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb49d6f-4fd3-4f0a-81e8-3d563a0f0da6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:26.211225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dfb49d6f-4fd3-4f0a-81e8-3d563a0f0da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:26.221423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:26.316867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb49d6f-4fd3-4f0a-81e8-3d563a0f0da6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:26.390411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:26.391053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:26.391343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:26.391627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:26.393305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:26.455283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:26.501424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:26.502899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff68e30-1a53-481e-8dbc-5a639400f455 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:26.521505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 672/1317] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:29.365039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7c278-a28b-4df2-9ad4-11c708f048d4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:29.365476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7c278-a28b-4df2-9ad4-11c708f048d4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:29.365676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7c278-a28b-4df2-9ad4-11c708f048d4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:29.366979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 646/1318] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:29.372211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05702-2406-441e-91aa-828788cb6513 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:29.372633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05702-2406-441e-91aa-828788cb6513 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:29.372808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05702-2406-441e-91aa-828788cb6513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:29.445649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-63c05702-2406-441e-91aa-828788cb6513 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:29.446877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-63c05702-2406-441e-91aa-828788cb6513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:29.542748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 673/1319] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:29.582477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7c278-a28b-4df2-9ad4-11c708f048d4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:29.584380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-15c7c278-a28b-4df2-9ad4-11c708f048d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:29.687432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7c278-a28b-4df2-9ad4-11c708f048d4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:29.759985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:29.760620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:29.760930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:29.761207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:29.762855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:29.770125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:29.869898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:29.871245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157d088-d1c7-4d12-bdb6-e33e3ad7d9d2 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:29.888935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 647/1320] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:33.108867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6da8-1346-4501-8b83-2b36974b6cf2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:33.109421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6da8-1346-4501-8b83-2b36974b6cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:33.109421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6da8-1346-4501-8b83-2b36974b6cf2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:33.110805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 674/1321] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:33.116191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692eaf5-79cc-4b1e-84ff-de0a60a973a1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:33.116644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692eaf5-79cc-4b1e-84ff-de0a60a973a1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:33.116762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692eaf5-79cc-4b1e-84ff-de0a60a973a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:33.181412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2692eaf5-79cc-4b1e-84ff-de0a60a973a1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:33.182607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2692eaf5-79cc-4b1e-84ff-de0a60a973a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:33.239782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:33.281682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 648/1322] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:33.318788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6da8-1346-4501-8b83-2b36974b6cf2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:33.320318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3d5b6da8-1346-4501-8b83-2b36974b6cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:33.339788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:33.431418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6da8-1346-4501-8b83-2b36974b6cf2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:33.512844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:33.513504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:33.513791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:33.514070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:33.515786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:33.593841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:33.634743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:33.636096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0caa-55d7-436b-a9f9-5b67fda83581 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:33.655365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 675/1323] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:35.772096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274573-d00d-48cf-9d1b-a5fd7b48b399 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.772629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274573-d00d-48cf-9d1b-a5fd7b48b399 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.772763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274573-d00d-48cf-9d1b-a5fd7b48b399 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.774091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 649/1324] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:35.779377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747e131-4d60-44cf-a18a-1326d646e600 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.779804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747e131-4d60-44cf-a18a-1326d646e600 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.780019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747e131-4d60-44cf-a18a-1326d646e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.845203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0747e131-4d60-44cf-a18a-1326d646e600 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:35.846448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0747e131-4d60-44cf-a18a-1326d646e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:35.945546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 676/1325] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:35.983020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274573-d00d-48cf-9d1b-a5fd7b48b399 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.984980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5f274573-d00d-48cf-9d1b-a5fd7b48b399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:36.104488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274573-d00d-48cf-9d1b-a5fd7b48b399 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:36.180612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:36.181290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:36.181578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:36.181857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:36.183514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:36.202564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:36.357736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:36.360606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014e791-4802-4c8f-9d28-895f343f25e4 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:36.399672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 650/1326] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:38.518358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2535-dfe2-42ad-a6cf-fd053c71c47c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:38.518864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2535-dfe2-42ad-a6cf-fd053c71c47c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:38.518864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2535-dfe2-42ad-a6cf-fd053c71c47c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:38.520263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 677/1327] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:38.525841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8747-2ce6-4b1f-8965-277c724d3b1c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:38.526324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8747-2ce6-4b1f-8965-277c724d3b1c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:38.526442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8747-2ce6-4b1f-8965-277c724d3b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:38.599184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-33ff8747-2ce6-4b1f-8965-277c724d3b1c None None] MFA Rules not processed for user `4b3ba696b60e4dc9b69780cc894e58a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:38.600565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-33ff8747-2ce6-4b1f-8965-277c724d3b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:38.667718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:38.704129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 651/1328] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:38.742424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab2535-dfe2-42ad-a6cf-fd053c71c47c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:38.743978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1dab2535-dfe2-42ad-a6cf-fd053c71c47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:38.777434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:38.854816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab2535-dfe2-42ad-a6cf-fd053c71c47c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:38.928148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:38.928831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:38.929118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:38.929399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:38.931025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:39.011136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:39.046499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:39.048012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690dbb87-e622-43a2-bad7-4fbeefce4b20 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:39.067866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 678/1329] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:41.757689 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299284-7e43-4aa4-9c72-0d260b0ebfad None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.758258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299284-7e43-4aa4-9c72-0d260b0ebfad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.758352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299284-7e43-4aa4-9c72-0d260b0ebfad None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.759699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 652/1330] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:41.765057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff1944-593d-48d5-98f8-0e17f16d64ad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.765450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff1944-593d-48d5-98f8-0e17f16d64ad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.765641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff1944-593d-48d5-98f8-0e17f16d64ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.830899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c9ff1944-593d-48d5-98f8-0e17f16d64ad None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:41.832032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c9ff1944-593d-48d5-98f8-0e17f16d64ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:41.927073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 679/1331] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:41.964271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299284-7e43-4aa4-9c72-0d260b0ebfad None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:41.965859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dc299284-7e43-4aa4-9c72-0d260b0ebfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:42.069594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299284-7e43-4aa4-9c72-0d260b0ebfad None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:42.142526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:42.143095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:42.143380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:42.143657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:42.145360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:42.171191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:42.254272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:42.256323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c2095-5e58-485e-9093-6dce2a38e3cf service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:42.274653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 653/1332] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:44.330116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c4aa-4ba4-4803-82bf-a7797733b927 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:44.330662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c4aa-4ba4-4803-82bf-a7797733b927 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:44.330662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c4aa-4ba4-4803-82bf-a7797733b927 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:44.332054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 680/1333] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:44.338044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0bb69-b49d-47f4-b194-399c9d47ca1c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:44.338468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0bb69-b49d-47f4-b194-399c9d47ca1c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:44.338606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0bb69-b49d-47f4-b194-399c9d47ca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:44.401897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3ca0bb69-b49d-47f4-b194-399c9d47ca1c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:44.403091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ca0bb69-b49d-47f4-b194-399c9d47ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:44.476909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:44.499194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 654/1334] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:44.539798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c4aa-4ba4-4803-82bf-a7797733b927 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:44.541379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c68c4aa-4ba4-4803-82bf-a7797733b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:44.581370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:44.647335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c4aa-4ba4-4803-82bf-a7797733b927 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:44.726970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:44.727614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:44.727901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:44.728178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:44.729860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:44.813782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:44.840739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:44.842198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d8f86-675b-4e03-b5f6-0dd583ed504c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:44.860346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 681/1335] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:47.545141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de8b76-6e86-4ba7-807c-51a8aa7d4f9c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:47.545665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de8b76-6e86-4ba7-807c-51a8aa7d4f9c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:47.545665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de8b76-6e86-4ba7-807c-51a8aa7d4f9c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:47.547012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 655/1336] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:47.552185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303b73-fdf6-4d1d-be77-a5387175cf4b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:47.553638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303b73-fdf6-4d1d-be77-a5387175cf4b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:47.553638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303b73-fdf6-4d1d-be77-a5387175cf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:47.618253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d4303b73-fdf6-4d1d-be77-a5387175cf4b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:47.619465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d4303b73-fdf6-4d1d-be77-a5387175cf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:47.717744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 682/1337] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:47.754472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de8b76-6e86-4ba7-807c-51a8aa7d4f9c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:47.756150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f4de8b76-6e86-4ba7-807c-51a8aa7d4f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:47.859761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de8b76-6e86-4ba7-807c-51a8aa7d4f9c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:47.932600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:47.933269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:47.933555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:47.933834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:47.935477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:47.971278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:48.043431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:48.044926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d842ec-c273-43a0-9a8b-edc86cb5017c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:48.069691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 656/1338] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:52.480427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa93fa6-b8c9-4dfc-bf57-d086440e14eb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:52.481015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa93fa6-b8c9-4dfc-bf57-d086440e14eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:52.481015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa93fa6-b8c9-4dfc-bf57-d086440e14eb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:52.482400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 683/1339] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:52.488130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95666940-7663-4e69-9e7a-bf5b1309862f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:52.489020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95666940-7663-4e69-9e7a-bf5b1309862f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:52.489020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95666940-7663-4e69-9e7a-bf5b1309862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:52.557033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-95666940-7663-4e69-9e7a-bf5b1309862f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:52.558266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-95666940-7663-4e69-9e7a-bf5b1309862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:52.660985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:52.674498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 657/1340] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:52.715073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa93fa6-b8c9-4dfc-bf57-d086440e14eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:52.716762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ffa93fa6-b8c9-4dfc-bf57-d086440e14eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:52.769793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:52.835797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa93fa6-b8c9-4dfc-bf57-d086440e14eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:52.916321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:52.917011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:52.917301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:52.917583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:52.919231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:53.011900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:53.028113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:53.029595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22c13e-8647-4c0d-ae52-8d0aa6c2802a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:53.048612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 684/1341] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:56.318136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6ba8e-e6a4-42f5-9e24-cb6b85405161 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:56.318777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6ba8e-e6a4-42f5-9e24-cb6b85405161 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:56.318777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6ba8e-e6a4-42f5-9e24-cb6b85405161 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:56.320231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 658/1342] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:34:56.325273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d1780-f4c5-4c59-ba89-54a6c7799cf5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:56.325763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d1780-f4c5-4c59-ba89-54a6c7799cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:56.325906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d1780-f4c5-4c59-ba89-54a6c7799cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:56.396646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bb6d1780-f4c5-4c59-ba89-54a6c7799cf5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:56.397867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bb6d1780-f4c5-4c59-ba89-54a6c7799cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:56.495192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 685/1343] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:34:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:56.532702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6ba8e-e6a4-42f5-9e24-cb6b85405161 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:56.534272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-75a6ba8e-e6a4-42f5-9e24-cb6b85405161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:56.637274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6ba8e-e6a4-42f5-9e24-cb6b85405161 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:56.712187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:56.712786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:56.713075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:56.713353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:56.715010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:56.760016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:56.831346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:56.832811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11bba9-95b1-406c-b6a9-3eeae5133523 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:56.853083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 659/1344] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:01.305176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb7da9-2ad4-44d5-bb83-134f6a1c8ef0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:01.305809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb7da9-2ad4-44d5-bb83-134f6a1c8ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:01.305809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb7da9-2ad4-44d5-bb83-134f6a1c8ef0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:01.307050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 686/1345] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:01.312395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695a20c-2d11-4be5-ac0d-6f0b0513deb5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:01.312910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695a20c-2d11-4be5-ac0d-6f0b0513deb5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:01.313009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695a20c-2d11-4be5-ac0d-6f0b0513deb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:01.382814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1695a20c-2d11-4be5-ac0d-6f0b0513deb5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:01.384054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1695a20c-2d11-4be5-ac0d-6f0b0513deb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:01.501879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:01.508728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 660/1346] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:01.544207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb7da9-2ad4-44d5-bb83-134f6a1c8ef0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:01.545860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5eb7da9-2ad4-44d5-bb83-134f6a1c8ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:01.617271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:01.664586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb7da9-2ad4-44d5-bb83-134f6a1c8ef0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:01.741039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:01.741706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:01.741990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:01.742271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:01.743963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:01.859844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:01.863474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:01.864996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea14c38-9e53-4b31-a31e-94662e6091ce service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:01.882703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 687/1347] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:06.986145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fdcf1-1d96-4542-83ef-5b2719899e3d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:06.986641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fdcf1-1d96-4542-83ef-5b2719899e3d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:06.986641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fdcf1-1d96-4542-83ef-5b2719899e3d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:06.988427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 661/1348] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:07.002738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf455-6582-4853-b4c8-b1afda05505d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:07.003755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf455-6582-4853-b4c8-b1afda05505d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:07.004262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf455-6582-4853-b4c8-b1afda05505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:07.089440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cdcaf455-6582-4853-b4c8-b1afda05505d None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:07.090939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cdcaf455-6582-4853-b4c8-b1afda05505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:07.216832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 688/1349] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:11.284543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb5fc4-860e-43d0-ad6b-9543b64e36f7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:11.285115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb5fc4-860e-43d0-ad6b-9543b64e36f7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:11.285491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb5fc4-860e-43d0-ad6b-9543b64e36f7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:11.288222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 662/1350] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:35:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:11.293735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d9f17-f354-480e-8e20-37978af99708 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:11.294200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d9f17-f354-480e-8e20-37978af99708 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:11.294518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d9f17-f354-480e-8e20-37978af99708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:11.361000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:11.383999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7b3d9f17-f354-480e-8e20-37978af99708 None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:11.385275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7b3d9f17-f354-480e-8e20-37978af99708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:11.496587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 689/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:35:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:12.875633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dabb1-0431-4fd3-acad-aaeda026b219 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:12.876180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dabb1-0431-4fd3-acad-aaeda026b219 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:12.876408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dabb1-0431-4fd3-acad-aaeda026b219 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:12.877832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 663/1352] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:12.883856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b06f0-fb59-4a3a-a161-7bf0bd6ab558 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:12.884317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b06f0-fb59-4a3a-a161-7bf0bd6ab558 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:12.884539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b06f0-fb59-4a3a-a161-7bf0bd6ab558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:12.962563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d18b06f0-fb59-4a3a-a161-7bf0bd6ab558 None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:12.963775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d18b06f0-fb59-4a3a-a161-7bf0bd6ab558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:13.009324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:13.077198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 690/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:35:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:16.512763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc567-9212-4a5f-8463-b37630d44a45 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:16.513924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc567-9212-4a5f-8463-b37630d44a45 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:16.514261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc567-9212-4a5f-8463-b37630d44a45 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:16.515877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 664/1354] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:16.522444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a270a0-4d20-400c-b242-f8ae4e2ed851 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:16.523521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a270a0-4d20-400c-b242-f8ae4e2ed851 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:16.524027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a270a0-4d20-400c-b242-f8ae4e2ed851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:16.614917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a0a270a0-4d20-400c-b242-f8ae4e2ed851 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:16.616872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a0a270a0-4d20-400c-b242-f8ae4e2ed851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:16.731070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 691/1355] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:16.784365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98de6-afb8-4971-874a-598695364c21 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:16.785018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98de6-afb8-4971-874a-598695364c21 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:16.785558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98de6-afb8-4971-874a-598695364c21 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:16.788257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 665/1356] 10.222.0.47 () {54 vars in 962 bytes} [Mon Jul 18 10:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:16.795559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5b857-32f3-4945-b940-4fb08d690545 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:16.805072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5b857-32f3-4945-b940-4fb08d690545 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:16.805072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5b857-32f3-4945-b940-4fb08d690545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:16.815384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:16.885653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0bd5b857-32f3-4945-b940-4fb08d690545 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:16.887030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0bd5b857-32f3-4945-b940-4fb08d690545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:17.008633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 692/1357] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Jul 18 10:35:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:17.017065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98de6-afb8-4971-874a-598695364c21 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:17.018687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1ca98de6-afb8-4971-874a-598695364c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:17.174000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98de6-afb8-4971-874a-598695364c21 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:17.262755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:17.305670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:17.306775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:17.307194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:17.307470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:17.309222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:17.403542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:17.405159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fa343-7c34-4016-a384-fe8b539791a9 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:17.425347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 666/1358] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:19.694937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6009e26-daaa-422b-b691-80c756b840be None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:19.694937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6009e26-daaa-422b-b691-80c756b840be None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:19.694937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6009e26-daaa-422b-b691-80c756b840be None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:19.694937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 693/1359] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:19.700201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147dd6-c6e7-478d-a147-bbfdf0a8daad None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:19.700581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147dd6-c6e7-478d-a147-bbfdf0a8daad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:19.701249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147dd6-c6e7-478d-a147-bbfdf0a8daad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:19.782408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4f147dd6-c6e7-478d-a147-bbfdf0a8daad None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:19.783749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4f147dd6-c6e7-478d-a147-bbfdf0a8daad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:19.824249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:19.887099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 667/1360] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:19.932832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6009e26-daaa-422b-b691-80c756b840be None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:19.934607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f6009e26-daaa-422b-b691-80c756b840be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:20.007312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:20.107340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6009e26-daaa-422b-b691-80c756b840be None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:20.253432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:20.253432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:20.253432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:20.253432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:20.265374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:20.297540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:20.371173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:20.372842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14baa-557d-4769-ae79-aa6eff7d839c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:20.392120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 694/1361] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:22.997460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b9680-6585-4036-990b-4c3f1940a30c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:22.998080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b9680-6585-4036-990b-4c3f1940a30c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:22.998080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b9680-6585-4036-990b-4c3f1940a30c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:22.999462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 668/1362] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:23.005788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd9889-b307-4565-8052-000ec82a5793 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:23.006246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd9889-b307-4565-8052-000ec82a5793 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:23.006339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd9889-b307-4565-8052-000ec82a5793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:23.076879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2ffd9889-b307-4565-8052-000ec82a5793 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:23.078162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2ffd9889-b307-4565-8052-000ec82a5793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:23.179618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:23.189069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 695/1363] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:23.232597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d0c7-a5e8-44fa-bebb-ca4baf790698 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:23.233279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d0c7-a5e8-44fa-bebb-ca4baf790698 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:23.233631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d0c7-a5e8-44fa-bebb-ca4baf790698 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:23.235080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 669/1364] 10.222.0.47 () {54 vars in 962 bytes} [Mon Jul 18 10:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:23.241285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4b20-3396-4a96-95af-0386036d3177 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:23.243109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4b20-3396-4a96-95af-0386036d3177 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:23.243109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4b20-3396-4a96-95af-0386036d3177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:23.334636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bd3c4b20-3396-4a96-95af-0386036d3177 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:23.335890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bd3c4b20-3396-4a96-95af-0386036d3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:23.464248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 696/1365] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Jul 18 10:35:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:23.471787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7d0c7-a5e8-44fa-bebb-ca4baf790698 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:23.474120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b2f7d0c7-a5e8-44fa-bebb-ca4baf790698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:23.629886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7d0c7-a5e8-44fa-bebb-ca4baf790698 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:23.651103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:23.746594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:23.747244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:23.747534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:23.747804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:23.749503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:23.845066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:23.846666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3f11b-7d41-4749-912a-d0def7e56b96 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:23.865353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 670/1366] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:24.834709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07376a8-566b-4989-ac2c-94a568a4ff9a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:24.835323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07376a8-566b-4989-ac2c-94a568a4ff9a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:24.835323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07376a8-566b-4989-ac2c-94a568a4ff9a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:24.836810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 697/1367] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:24.842094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393a79f-b101-4bc6-8120-2d1e5db3821c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:24.842512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393a79f-b101-4bc6-8120-2d1e5db3821c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:24.842735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393a79f-b101-4bc6-8120-2d1e5db3821c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:24.874708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:24.911559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5393a79f-b101-4bc6-8120-2d1e5db3821c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:24.912963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5393a79f-b101-4bc6-8120-2d1e5db3821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:25.016417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 671/1368] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:25.392459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9263e-877e-4616-a174-b84eb832433d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:25.392899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9263e-877e-4616-a174-b84eb832433d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:25.393284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9263e-877e-4616-a174-b84eb832433d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:25.396538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 698/1369] 10.222.0.47 () {54 vars in 964 bytes} [Mon Jul 18 10:35:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:25.401504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25c25b-5f15-4193-bbd9-b454cd0be88b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:25.401796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25c25b-5f15-4193-bbd9-b454cd0be88b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:25.402082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25c25b-5f15-4193-bbd9-b454cd0be88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:25.485597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dc25c25b-5f15-4193-bbd9-b454cd0be88b None None] MFA Rules not processed for user `bd8f56a4b2d94ec1aec6f0d044cbc1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:25.486862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dc25c25b-5f15-4193-bbd9-b454cd0be88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:25.537841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:25.572845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 672/1370] 10.222.0.47 () {58 vars in 1061 bytes} [Mon Jul 18 10:35:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:25.577781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9263e-877e-4616-a174-b84eb832433d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:25.579298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cb9263e-877e-4616-a174-b84eb832433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:25.637073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:25.694288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9263e-877e-4616-a174-b84eb832433d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:25.769994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:25.770667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:25.771035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:25.771319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:25.773005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:25.878603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:25.901193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:25.902488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f4df7-b308-40f5-ae8c-baabfb01e4f4 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:25.924418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 699/1371] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:35:26.211922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a11a3e-ff65-4144-84af-23282a1fcf24 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:26.213061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a11a3e-ff65-4144-84af-23282a1fcf24 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:26.213061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a11a3e-ff65-4144-84af-23282a1fcf24 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:26.219113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd58fb-df9e-498b-85e2-83e4d4c1efbd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:26.219431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd58fb-df9e-498b-85e2-83e4d4c1efbd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:26.220038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd58fb-df9e-498b-85e2-83e4d4c1efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:26.223870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 673/1372] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:26.294767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c4fd58fb-df9e-498b-85e2-83e4d4c1efbd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:26.295963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4fd58fb-df9e-498b-85e2-83e4d4c1efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:26.398681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 700/1373] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:26.446890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a11a3e-ff65-4144-84af-23282a1fcf24 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:26.448424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-91a11a3e-ff65-4144-84af-23282a1fcf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:26.600626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a11a3e-ff65-4144-84af-23282a1fcf24 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:26.655378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:26.723092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:26.723891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:26.728509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:26.728509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:26.728509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:26.819924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:26.821560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5031ff-b350-4f1e-a702-e6e4fa3dd495 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:26.840781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 674/1374] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:29.187877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb504df-c03c-4c64-8592-f6a37c9ab70b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:29.188402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb504df-c03c-4c64-8592-f6a37c9ab70b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:29.188402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb504df-c03c-4c64-8592-f6a37c9ab70b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:29.189796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 701/1375] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:29.195853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cb4f-1574-4055-961e-8aeb9319a720 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:29.196261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cb4f-1574-4055-961e-8aeb9319a720 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:29.196415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cb4f-1574-4055-961e-8aeb9319a720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:29.261481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6e71cb4f-1574-4055-961e-8aeb9319a720 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:29.262710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6e71cb4f-1574-4055-961e-8aeb9319a720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:29.274650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:29.366865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 675/1376] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:29.413759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb504df-c03c-4c64-8592-f6a37c9ab70b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:29.415317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-acb504df-c03c-4c64-8592-f6a37c9ab70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:29.502873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:29.654817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb504df-c03c-4c64-8592-f6a37c9ab70b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:29.812363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:29.813055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:29.813342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:29.813622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:29.815258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:29.840623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:29.920559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:29.922574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3c745-8a6d-4e20-91b9-4014992678e4 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:29.946044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 702/1377] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:32.296981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d4787-4f2e-46b4-a279-d9dbdf0dbf4c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:32.297534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d4787-4f2e-46b4-a279-d9dbdf0dbf4c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:32.297534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d4787-4f2e-46b4-a279-d9dbdf0dbf4c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:32.299086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 676/1378] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:32.304354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4de1-5fad-4246-bb27-8a330968e1bb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:32.304806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4de1-5fad-4246-bb27-8a330968e1bb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:32.304975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4de1-5fad-4246-bb27-8a330968e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:32.374438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0e6f4de1-5fad-4246-bb27-8a330968e1bb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:32.375698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e6f4de1-5fad-4246-bb27-8a330968e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:32.474723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:32.481243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 703/1379] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:32.526198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d4787-4f2e-46b4-a279-d9dbdf0dbf4c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:32.527711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-210d4787-4f2e-46b4-a279-d9dbdf0dbf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:32.687614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:32.700217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d4787-4f2e-46b4-a279-d9dbdf0dbf4c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:32.825549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:32.826200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:32.826486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:32.826765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:32.828391 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:32.930139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:32.931922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b55983-47d0-4e95-8944-51855a398b46 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:32.950256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:32.959818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 677/1380] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:35.235362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3939f2-ca3b-4d20-9041-c6531eaa6815 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:35.236071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3939f2-ca3b-4d20-9041-c6531eaa6815 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:35.236071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3939f2-ca3b-4d20-9041-c6531eaa6815 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:35.237427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 704/1381] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:35.242873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4c3cb-a998-4478-857c-91e746d67306 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:35.243390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4c3cb-a998-4478-857c-91e746d67306 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:35.243390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4c3cb-a998-4478-857c-91e746d67306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:35.321459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-60b4c3cb-a998-4478-857c-91e746d67306 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:35.323374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-60b4c3cb-a998-4478-857c-91e746d67306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:35.442488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 678/1382] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:35.486273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3939f2-ca3b-4d20-9041-c6531eaa6815 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:35.489049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2c3939f2-ca3b-4d20-9041-c6531eaa6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:35.653542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3939f2-ca3b-4d20-9041-c6531eaa6815 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:35.724322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:35.777910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:35.778568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:35.778857 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:35.779139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:35.780832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:35.872232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:35.873866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ae471-30e1-45e2-8637-dfd05acc0347 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:35.892371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 705/1383] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:37.932135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4442b-1b4c-4feb-9a9b-733ed00137a7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:37.932640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4442b-1b4c-4feb-9a9b-733ed00137a7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:37.932640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4442b-1b4c-4feb-9a9b-733ed00137a7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:37.934117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 679/1384] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:37.940408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fc7da-7979-48a8-91ec-1c9a614dfe8f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:37.941385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fc7da-7979-48a8-91ec-1c9a614dfe8f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:37.941574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fc7da-7979-48a8-91ec-1c9a614dfe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:38.008877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-797fc7da-7979-48a8-91ec-1c9a614dfe8f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:38.010228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-797fc7da-7979-48a8-91ec-1c9a614dfe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:38.046826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:38.115831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 706/1385] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:38.171265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4442b-1b4c-4feb-9a9b-733ed00137a7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:38.173367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-39d4442b-1b4c-4feb-9a9b-733ed00137a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:38.269652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:38.357523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4442b-1b4c-4feb-9a9b-733ed00137a7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:38.479607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:38.480266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:38.480567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:38.480884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:38.482605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:38.524486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:38.577587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:38.579227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31eca12-b1f6-42a8-9f76-a0b45b0ca8ee service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:38.598293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 680/1386] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:40.789700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43027b0-53c6-46bd-b5ac-a346e346de0b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:40.790211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43027b0-53c6-46bd-b5ac-a346e346de0b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:40.790211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43027b0-53c6-46bd-b5ac-a346e346de0b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:40.791749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 707/1387] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:40.796883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4884a5e-96f0-4483-b818-46e6f129bf8e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:40.797333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4884a5e-96f0-4483-b818-46e6f129bf8e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:40.797450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4884a5e-96f0-4483-b818-46e6f129bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:40.866338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e4884a5e-96f0-4483-b818-46e6f129bf8e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:40.867541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e4884a5e-96f0-4483-b818-46e6f129bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:40.963561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 681/1388] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:41.009853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43027b0-53c6-46bd-b5ac-a346e346de0b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:41.011596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e43027b0-53c6-46bd-b5ac-a346e346de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:41.161104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43027b0-53c6-46bd-b5ac-a346e346de0b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:41.177684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:41.288893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:41.289412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:41.289702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:41.289983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:41.291753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:41.389997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:41.391612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a70c12-d862-4856-abc4-5bb6ea54de03 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:41.411705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 708/1389] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:35:41.554489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5164-f694-4829-afad-6bc80997ffa1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:41.554859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5164-f694-4829-afad-6bc80997ffa1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:41.555210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5164-f694-4829-afad-6bc80997ffa1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:41.556669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 682/1390] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:41.562315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b8e0e-a754-4f68-b6c3-1847d798aaf5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:41.562628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b8e0e-a754-4f68-b6c3-1847d798aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:41.562910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b8e0e-a754-4f68-b6c3-1847d798aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:41.587998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:41.627999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-740b8e0e-a754-4f68-b6c3-1847d798aaf5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:41.629283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-740b8e0e-a754-4f68-b6c3-1847d798aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:41.732081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 709/1391] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:41.789899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a78c-0e80-46b1-96e4-92cb0392aa11 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:41.790191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a78c-0e80-46b1-96e4-92cb0392aa11 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:41.790476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a78c-0e80-46b1-96e4-92cb0392aa11 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:41.791877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 683/1392] 10.222.0.47 () {54 vars in 961 bytes} [Mon Jul 18 10:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:41.797599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d08f3-61d4-4d62-98e9-211c2f3cca02 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:41.798140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d08f3-61d4-4d62-98e9-211c2f3cca02 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:41.798235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d08f3-61d4-4d62-98e9-211c2f3cca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:41.879376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-df0d08f3-61d4-4d62-98e9-211c2f3cca02 None None] MFA Rules not processed for user `aefb3626f45544ba81603a722821fce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:41.880601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df0d08f3-61d4-4d62-98e9-211c2f3cca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:41.922027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:41.958706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 710/1393] 10.222.0.47 () {58 vars in 1058 bytes} [Mon Jul 18 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:41.965907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67a78c-0e80-46b1-96e4-92cb0392aa11 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:41.967421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2d67a78c-0e80-46b1-96e4-92cb0392aa11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:41.982006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:42.083665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67a78c-0e80-46b1-96e4-92cb0392aa11 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:42.158498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:42.159142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:42.159477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:42.159789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:42.161558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:42.229117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:42.269326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:42.270902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568fc78-c580-4b7c-9b09-39ebf8ca8154 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:42.289642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 684/1394] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:43.185484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1218-f6ae-4d5b-9135-33afa2aa4d82 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:43.185963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1218-f6ae-4d5b-9135-33afa2aa4d82 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:43.186069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1218-f6ae-4d5b-9135-33afa2aa4d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:43.252147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a26a1218-f6ae-4d5b-9135-33afa2aa4d82 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:43.253392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a26a1218-f6ae-4d5b-9135-33afa2aa4d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:43.346725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 711/1395] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:43.392980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90a209-2357-4d89-b930-289f56934ec1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:43.393466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90a209-2357-4d89-b930-289f56934ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:43.393576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90a209-2357-4d89-b930-289f56934ec1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:43.395528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 685/1396] 10.222.0.47 () {54 vars in 943 bytes} [Mon Jul 18 10:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:43.402008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28c8a3-df46-451f-9959-7a9b7cd597f2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:43.402333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28c8a3-df46-451f-9959-7a9b7cd597f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:43.402620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28c8a3-df46-451f-9959-7a9b7cd597f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:43.474655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:43.482162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fb28c8a3-df46-451f-9959-7a9b7cd597f2 None None] MFA Rules not processed for user `5554e4999dc04da682c7f1cbd5f75abc`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:43.489072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fb28c8a3-df46-451f-9959-7a9b7cd597f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:43.589109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 712/1397] 10.222.0.47 () {58 vars in 1040 bytes} [Mon Jul 18 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:43.598641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90a209-2357-4d89-b930-289f56934ec1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:43.600137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2c90a209-2357-4d89-b930-289f56934ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:43.719941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f137b6-af40-48bb-a4dd-caee91159bb5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:43.720236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f137b6-af40-48bb-a4dd-caee91159bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:43.720535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f137b6-af40-48bb-a4dd-caee91159bb5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:43.722656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 713/1398] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:35:43.727605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:43.727896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:43.728179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:43.740194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90a209-2357-4d89-b930-289f56934ec1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:43.778346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:43.794904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:43.796119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:43.847847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:43.848494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:43.848862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:43.849094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:43.850792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:43.858964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:43.894346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 714/1399] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:43.933333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:43.934945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770bf19-7a87-4c25-b865-423afac1e843 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:43.937953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:43.939190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:43.947860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 686/1400] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:35:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:35:44.054771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0e997-264a-4a8a-a6bc-e9e7242518c5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:44.129442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:44.168256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:44.168943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:44.169228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:44.169503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:44.171137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:44.267479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:44.268655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b50c4-984c-4188-b513-adff851ee177 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:44.288063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 715/1401] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:35:46.574719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b88925-1c1c-4983-a27e-4481464dd338 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.575279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b88925-1c1c-4983-a27e-4481464dd338 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.575279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b88925-1c1c-4983-a27e-4481464dd338 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.576737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 687/1402] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:46.582471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417c5e6-a947-4556-921a-f4122cc78ab2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.583220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417c5e6-a947-4556-921a-f4122cc78ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.583316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417c5e6-a947-4556-921a-f4122cc78ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.651057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d417c5e6-a947-4556-921a-f4122cc78ab2 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:46.652405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d417c5e6-a947-4556-921a-f4122cc78ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:46.699218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:46.763859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 716/1403] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:46.810010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b88925-1c1c-4983-a27e-4481464dd338 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:46.811572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-43b88925-1c1c-4983-a27e-4481464dd338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:46.892391 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:46.978106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b88925-1c1c-4983-a27e-4481464dd338 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:47.108225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:47.108910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:47.109202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:47.109487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:47.111145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:47.145541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:47.219253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:47.223719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b594-1ca5-4d35-a8f1-f3fc62615ba3 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:47.244176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 688/1404] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:49.492584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759652f-588c-457a-b74b-7ffdd0c15a83 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:49.493134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759652f-588c-457a-b74b-7ffdd0c15a83 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:49.493244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759652f-588c-457a-b74b-7ffdd0c15a83 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:49.494877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 717/1405] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:49.500075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c889-fb77-44a9-a481-f03bbb0f607c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:49.500516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c889-fb77-44a9-a481-f03bbb0f607c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:49.500757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c889-fb77-44a9-a481-f03bbb0f607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:49.573520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8aa0c889-fb77-44a9-a481-f03bbb0f607c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:49.574752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8aa0c889-fb77-44a9-a481-f03bbb0f607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:49.695167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 689/1406] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:49.745904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759652f-588c-457a-b74b-7ffdd0c15a83 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:49.749627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f759652f-588c-457a-b74b-7ffdd0c15a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:49.908807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759652f-588c-457a-b74b-7ffdd0c15a83 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:49.932670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:50.041854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:50.042517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:50.042803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:50.043083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:50.044798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:50.145444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:50.147412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78b039-ed5c-418f-872a-e6005d8521fa service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:50.169112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 718/1407] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:59.692896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:00.164500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:08.027046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961e834-99de-4fad-8624-ee98eeb4a5f6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:08.027628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961e834-99de-4fad-8624-ee98eeb4a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:08.027628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961e834-99de-4fad-8624-ee98eeb4a5f6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:08.029069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 690/1408] 10.222.0.47 () {54 vars in 918 bytes} [Mon Jul 18 10:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:08.035026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadd429-3bed-42ef-912a-9408a0f2abcd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:08.035471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadd429-3bed-42ef-912a-9408a0f2abcd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:08.035582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadd429-3bed-42ef-912a-9408a0f2abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:08.110508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1cadd429-3bed-42ef-912a-9408a0f2abcd None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:08.112490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1cadd429-3bed-42ef-912a-9408a0f2abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:08.218300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 719/1409] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jul 18 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:08.233274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039bff0-c7d6-490a-ae2f-05c9e7f8fd55 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:08.233566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039bff0-c7d6-490a-ae2f-05c9e7f8fd55 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:08.233850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039bff0-c7d6-490a-ae2f-05c9e7f8fd55 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:08.235228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 691/1410] 10.222.0.47 () {54 vars in 918 bytes} [Mon Jul 18 10:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:08.240611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49853b-cc11-4dea-be55-0abf91eb3885 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:08.240936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49853b-cc11-4dea-be55-0abf91eb3885 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:08.241223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49853b-cc11-4dea-be55-0abf91eb3885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:08.313325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:08.316173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1a49853b-cc11-4dea-be55-0abf91eb3885 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:08.317421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1a49853b-cc11-4dea-be55-0abf91eb3885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:08.432784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 720/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jul 18 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:10.923465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef8438-ae46-4f70-88d3-f130b73830f7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.924132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef8438-ae46-4f70-88d3-f130b73830f7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.924132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef8438-ae46-4f70-88d3-f130b73830f7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.925451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 692/1412] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:10.931540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ad98-7d72-4ad4-8a63-5ebf8c215b82 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.931983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ad98-7d72-4ad4-8a63-5ebf8c215b82 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.932182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ad98-7d72-4ad4-8a63-5ebf8c215b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.998254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7750ad98-7d72-4ad4-8a63-5ebf8c215b82 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:10.999473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7750ad98-7d72-4ad4-8a63-5ebf8c215b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:11.053917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:11.095900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 721/1413] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:11.138073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef8438-ae46-4f70-88d3-f130b73830f7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.139599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66ef8438-ae46-4f70-88d3-f130b73830f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:11.288138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef8438-ae46-4f70-88d3-f130b73830f7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.360361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:11.402185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.402839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.403126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.403403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.405104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:11.494995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.496593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edbec9-9f6f-47ab-abda-823217c12ec3 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.514495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 693/1414] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:12.441280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7159b-4285-4013-94b9-689852725f95 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.441810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7159b-4285-4013-94b9-689852725f95 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.441810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7159b-4285-4013-94b9-689852725f95 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.443215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 722/1415] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:12.448209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534cd38-6de3-4b42-b9a8-c209a43d049c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.448661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534cd38-6de3-4b42-b9a8-c209a43d049c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.448845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534cd38-6de3-4b42-b9a8-c209a43d049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.524469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d534cd38-6de3-4b42-b9a8-c209a43d049c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:12.525635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d534cd38-6de3-4b42-b9a8-c209a43d049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:12.552153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:12.621770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 694/1416] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:12.772569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7159b-4285-4013-94b9-689852725f95 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:12.774128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7dd7159b-4285-4013-94b9-689852725f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:12.921400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7159b-4285-4013-94b9-689852725f95 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:12.963135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:13.036039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:13.036739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.037023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.037293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.038932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:13.128353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:13.130005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c0d5f-28f8-4823-9741-286c2b9d13e8 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:13.148369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 723/1417] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:14.047810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6847-75c1-4935-bbbe-5fd6ba39f8ef None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.048394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6847-75c1-4935-bbbe-5fd6ba39f8ef None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.048394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6847-75c1-4935-bbbe-5fd6ba39f8ef None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.050521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 695/1418] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:14.055485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beef1af-292e-4cc5-a6cf-f64a1068a679 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.055957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beef1af-292e-4cc5-a6cf-f64a1068a679 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.056068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beef1af-292e-4cc5-a6cf-f64a1068a679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.109741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:14.131621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3beef1af-292e-4cc5-a6cf-f64a1068a679 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:14.132886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3beef1af-292e-4cc5-a6cf-f64a1068a679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:14.228966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 724/1419] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:14.381438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e6847-75c1-4935-bbbe-5fd6ba39f8ef None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.382889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-268e6847-75c1-4935-bbbe-5fd6ba39f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:14.540007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e6847-75c1-4935-bbbe-5fd6ba39f8ef None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.552638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:14.652436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.653037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.653328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.653608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.655258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:14.744312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.745942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbadb53-83fe-488e-9084-ac2cc6abf173 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.763651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 696/1420] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:15.630386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7e467-3817-4b85-b455-0974b6c68cf8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:15.630955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7e467-3817-4b85-b455-0974b6c68cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:15.631064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7e467-3817-4b85-b455-0974b6c68cf8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:15.632401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 725/1421] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:15.637434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23579554-a203-4ad7-b8e8-91838d43824f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:15.637827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23579554-a203-4ad7-b8e8-91838d43824f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:15.637989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23579554-a203-4ad7-b8e8-91838d43824f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:15.656260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:15.714055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-23579554-a203-4ad7-b8e8-91838d43824f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:15.715173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-23579554-a203-4ad7-b8e8-91838d43824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:15.810951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 697/1422] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:15.967735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7e467-3817-4b85-b455-0974b6c68cf8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:15.969301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59a7e467-3817-4b85-b455-0974b6c68cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:16.203055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:16.238503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7e467-3817-4b85-b455-0974b6c68cf8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:16.355305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:16.355953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:16.356237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:16.356520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:16.358178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:16.433619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:16.447333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:16.448961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577599d0-92fc-436e-ae6e-64f59488c64c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:16.467840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 726/1423] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:17.332255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e267-4980-49bd-9932-42592272cc6e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:17.333138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e267-4980-49bd-9932-42592272cc6e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:17.333138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e267-4980-49bd-9932-42592272cc6e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:17.334476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 698/1424] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:17.339392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a953d0-3080-4e15-8028-c02a4d2fff00 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:17.339791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a953d0-3080-4e15-8028-c02a4d2fff00 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:17.339962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a953d0-3080-4e15-8028-c02a4d2fff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:17.414257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d3a953d0-3080-4e15-8028-c02a4d2fff00 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:17.415468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d3a953d0-3080-4e15-8028-c02a4d2fff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:17.510873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 727/1425] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:17.663281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0e267-4980-49bd-9932-42592272cc6e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:17.664879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e3c0e267-4980-49bd-9932-42592272cc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:17.749327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:17.811282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0e267-4980-49bd-9932-42592272cc6e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:17.925287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:17.925931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:17.926212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:17.926483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:17.928120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:17.975865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:18.016267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:18.018523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92fc53-2dcc-4e4a-a90a-e5e2f2516aaf service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:18.036198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 699/1426] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:18.945415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2026e1-e077-4bc4-a3b0-11872723dd2e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:18.945969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2026e1-e077-4bc4-a3b0-11872723dd2e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:18.945969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2026e1-e077-4bc4-a3b0-11872723dd2e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:18.947338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 728/1427] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:18.952359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50767055-5e34-4640-ac8d-2e38f706bf8d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:18.952780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50767055-5e34-4640-ac8d-2e38f706bf8d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:18.952961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50767055-5e34-4640-ac8d-2e38f706bf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:19.030112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-50767055-5e34-4640-ac8d-2e38f706bf8d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:19.031335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-50767055-5e34-4640-ac8d-2e38f706bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:19.127693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 700/1428] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:19.280865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2026e1-e077-4bc4-a3b0-11872723dd2e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:19.282387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be2026e1-e077-4bc4-a3b0-11872723dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:19.330576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:19.430481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2026e1-e077-4bc4-a3b0-11872723dd2e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:19.543305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:19.543871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:19.544154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:19.544427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:19.546096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:19.553496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:19.635101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:19.636703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f7180-7bbd-4952-a158-af5ff4d59d52 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:19.654931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 729/1429] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:20.512207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f39996-e173-40a0-b557-04292f93cd64 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:20.512765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f39996-e173-40a0-b557-04292f93cd64 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:20.512765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f39996-e173-40a0-b557-04292f93cd64 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:20.514172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 701/1430] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:20.520003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9496d-50ef-41aa-bf45-17c5debeb615 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:20.520450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9496d-50ef-41aa-bf45-17c5debeb615 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:20.520569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9496d-50ef-41aa-bf45-17c5debeb615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:20.593320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-81b9496d-50ef-41aa-bf45-17c5debeb615 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:20.594523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81b9496d-50ef-41aa-bf45-17c5debeb615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:20.662516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:20.690649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 730/1431] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:20.844297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f39996-e173-40a0-b557-04292f93cd64 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:20.846485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-58f39996-e173-40a0-b557-04292f93cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:20.864095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:20.992010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f39996-e173-40a0-b557-04292f93cd64 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.079167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:21.104982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.105667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.105953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.106233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.107864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:21.198468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.200059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0e06c-ec4e-4820-924c-289723b28490 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.218161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 702/1432] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:22.098003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4edc-b518-46ec-bd7d-f33983c74eec None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:22.098545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4edc-b518-46ec-bd7d-f33983c74eec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:22.098545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4edc-b518-46ec-bd7d-f33983c74eec None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:22.099933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 731/1433] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:22.105868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee9c4-e13a-4d1a-a98c-ced5a140a0a0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:22.106273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee9c4-e13a-4d1a-a98c-ced5a140a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:22.106432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee9c4-e13a-4d1a-a98c-ced5a140a0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:22.183613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f51ee9c4-e13a-4d1a-a98c-ced5a140a0a0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:22.184871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f51ee9c4-e13a-4d1a-a98c-ced5a140a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:22.226610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:22.281240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 703/1434] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:22.433913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4edc-b518-46ec-bd7d-f33983c74eec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:22.435386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e4c4edc-b518-46ec-bd7d-f33983c74eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:22.580317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4edc-b518-46ec-bd7d-f33983c74eec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:22.633902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:22.692901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:22.693477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:22.693763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:22.694040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:22.695688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:22.784118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:22.785743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7fb33-38ad-4610-9bc5-e2fb6f3644a3 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:22.803920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 732/1435] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:23.717149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c05dc-c1cb-49c1-9c60-4893e133bebf None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:23.717715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c05dc-c1cb-49c1-9c60-4893e133bebf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:23.717715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c05dc-c1cb-49c1-9c60-4893e133bebf None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:23.719006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 704/1436] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:23.724504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e0bf-9898-4555-b124-e09b21e488eb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:23.724977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e0bf-9898-4555-b124-e09b21e488eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:23.725086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e0bf-9898-4555-b124-e09b21e488eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:23.800608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0f25e0bf-9898-4555-b124-e09b21e488eb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:23.801856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0f25e0bf-9898-4555-b124-e09b21e488eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:23.803895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:23.900057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 733/1437] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:24.051431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c05dc-c1cb-49c1-9c60-4893e133bebf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:24.053057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-037c05dc-c1cb-49c1-9c60-4893e133bebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:24.199905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c05dc-c1cb-49c1-9c60-4893e133bebf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:24.227252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:24.313260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:24.313842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:24.314254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:24.314537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:24.316186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:24.404610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:24.406214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dac837-fe7b-4d0d-8f8d-c94b0ea6dd98 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:24.424066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 705/1438] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:26.337343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797e6e-16e1-4647-a5b0-b8e077278b1b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:26.337910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797e6e-16e1-4647-a5b0-b8e077278b1b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:26.337910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797e6e-16e1-4647-a5b0-b8e077278b1b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:26.339263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 734/1439] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:26.344560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1025927-ff3d-4274-aaf2-452f7094079f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:26.345072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1025927-ff3d-4274-aaf2-452f7094079f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:26.345749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1025927-ff3d-4274-aaf2-452f7094079f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:26.385082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:26.411870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b1025927-ff3d-4274-aaf2-452f7094079f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:26.413172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b1025927-ff3d-4274-aaf2-452f7094079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:26.511676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 706/1440] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:26.550721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797e6e-16e1-4647-a5b0-b8e077278b1b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:26.554987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1797e6e-16e1-4647-a5b0-b8e077278b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:26.658464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797e6e-16e1-4647-a5b0-b8e077278b1b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:26.690740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:26.731183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:26.731823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:26.732102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:26.732376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:26.734820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:26.842678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:26.844025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dca328-da93-496c-8175-c383dd6cf831 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:26.862225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 735/1441] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:29.046241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cdeb6-8bac-4c4d-8269-2b9a0096896c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:29.046813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cdeb6-8bac-4c4d-8269-2b9a0096896c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:29.046813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cdeb6-8bac-4c4d-8269-2b9a0096896c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:29.049221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 707/1442] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:29.054564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefb6f4-3619-4d01-a8e1-2e5813b647f5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:29.054974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefb6f4-3619-4d01-a8e1-2e5813b647f5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:29.055158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefb6f4-3619-4d01-a8e1-2e5813b647f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:29.119896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2aefb6f4-3619-4d01-a8e1-2e5813b647f5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:29.121167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2aefb6f4-3619-4d01-a8e1-2e5813b647f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:29.134598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:29.219561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 736/1443] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:29.260111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cdeb6-8bac-4c4d-8269-2b9a0096896c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:29.261754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ea1cdeb6-8bac-4c4d-8269-2b9a0096896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:29.365040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cdeb6-8bac-4c4d-8269-2b9a0096896c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:29.377128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:29.438899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:29.439607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:29.439900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:29.440184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:29.441859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:29.564225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:29.565746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aada41-e566-4026-bb21-59527e8a1687 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:29.584580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 708/1444] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:32.012453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5cf4c-e936-4eb4-9bcd-dbf9a5b3a139 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:32.013046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5cf4c-e936-4eb4-9bcd-dbf9a5b3a139 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:32.013046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5cf4c-e936-4eb4-9bcd-dbf9a5b3a139 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:32.014502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 737/1445] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:32.020670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75660898-7a85-4371-b9fd-fc32c9d39892 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:32.021158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75660898-7a85-4371-b9fd-fc32c9d39892 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:32.021298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75660898-7a85-4371-b9fd-fc32c9d39892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:32.070005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:32.088403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-75660898-7a85-4371-b9fd-fc32c9d39892 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:32.089665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-75660898-7a85-4371-b9fd-fc32c9d39892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:32.187492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 709/1446] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:32.227820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5cf4c-e936-4eb4-9bcd-dbf9a5b3a139 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:32.229452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5cc5cf4c-e936-4eb4-9bcd-dbf9a5b3a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:32.336828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5cf4c-e936-4eb4-9bcd-dbf9a5b3a139 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:32.377467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:32.410866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:32.411514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:32.411801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:32.412079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:32.414565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:32.523151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:32.524501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934200d-9a77-4fb6-bf5a-6e5d55bfa4f0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:32.543933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 738/1447] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:34.696724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6365d-a061-4405-ac60-b6c1fc64be51 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:34.697273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6365d-a061-4405-ac60-b6c1fc64be51 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:34.697273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6365d-a061-4405-ac60-b6c1fc64be51 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:34.698600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 710/1448] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:34.703796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5c01-bf93-4d58-8722-b1300008aaa4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:34.704266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5c01-bf93-4d58-8722-b1300008aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:34.704384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5c01-bf93-4d58-8722-b1300008aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:34.767922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e84e5c01-bf93-4d58-8722-b1300008aaa4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:34.769157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e84e5c01-bf93-4d58-8722-b1300008aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:34.789454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:34.866721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 739/1449] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:34.920545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d37d9-c6f7-4890-8b57-0e9fbd4f92a6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:34.920870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d37d9-c6f7-4890-8b57-0e9fbd4f92a6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:34.921155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d37d9-c6f7-4890-8b57-0e9fbd4f92a6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:34.922537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 711/1450] 10.222.0.47 () {54 vars in 965 bytes} [Mon Jul 18 10:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:34.929304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61548f27-be66-4185-bf83-e2a4121335f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:34.929603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61548f27-be66-4185-bf83-e2a4121335f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:34.929887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61548f27-be66-4185-bf83-e2a4121335f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:35.006116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-61548f27-be66-4185-bf83-e2a4121335f9 None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:35.007299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61548f27-be66-4185-bf83-e2a4121335f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:35.081728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 740/1451] 10.222.0.47 () {58 vars in 1062 bytes} [Mon Jul 18 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:35.091107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d37d9-c6f7-4890-8b57-0e9fbd4f92a6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:35.092633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-673d37d9-c6f7-4890-8b57-0e9fbd4f92a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:35.193844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d37d9-c6f7-4890-8b57-0e9fbd4f92a6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:35.213936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:35.268850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:35.269506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:35.269792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:35.270073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:35.271728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:35.381590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:35.383024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ea19e-80ee-4484-88fb-c23699ab0bc1 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:35.401785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 712/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:37.973207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b71f7-fc48-4cd5-9f7a-a5549efdf8b8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:37.973773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b71f7-fc48-4cd5-9f7a-a5549efdf8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:37.973773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b71f7-fc48-4cd5-9f7a-a5549efdf8b8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:37.975158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:37.980339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7e58a-3718-4781-9291-bd8453f89027 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:37.980779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7e58a-3718-4781-9291-bd8453f89027 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:37.993345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7e58a-3718-4781-9291-bd8453f89027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:38.044953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:38.047645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d2e7e58a-3718-4781-9291-bd8453f89027 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:38.048931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d2e7e58a-3718-4781-9291-bd8453f89027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:38.151524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:38.189693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b71f7-fc48-4cd5-9f7a-a5549efdf8b8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:38.191226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b45b71f7-fc48-4cd5-9f7a-a5549efdf8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:38.209609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:38.304814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b71f7-fc48-4cd5-9f7a-a5549efdf8b8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:38.377969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:38.378634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:38.378918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:38.379193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:38.380892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:38.450718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:38.493085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:38.494527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91f25-23f9-4d0d-bb34-3a918d814c7d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:38.516035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 742/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:40.728302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eea8e-acd6-44cf-ae74-faa21b9f68fa None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:40.728874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eea8e-acd6-44cf-ae74-faa21b9f68fa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:40.728874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eea8e-acd6-44cf-ae74-faa21b9f68fa None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:40.731573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:40.737497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee5ef9-f278-4a55-845d-a1acceb62f8c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:40.737951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee5ef9-f278-4a55-845d-a1acceb62f8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:40.738078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee5ef9-f278-4a55-845d-a1acceb62f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:40.821402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f5ee5ef9-f278-4a55-845d-a1acceb62f8c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:40.822663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5ee5ef9-f278-4a55-845d-a1acceb62f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:40.930169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:40.971250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eea8e-acd6-44cf-ae74-faa21b9f68fa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:40.972927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e38eea8e-acd6-44cf-ae74-faa21b9f68fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:41.078875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eea8e-acd6-44cf-ae74-faa21b9f68fa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:41.107688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:41.152508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:41.153181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:41.153467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:41.153745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:41.155398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:41.264334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:41.265800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb16ff9-c0d5-4a81-9fc8-b82a6514873c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:41.284743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 715/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:36:43.383159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f2e04-68b0-4c03-84dd-c66036d156f3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:43.383698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f2e04-68b0-4c03-84dd-c66036d156f3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:43.383794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f2e04-68b0-4c03-84dd-c66036d156f3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:43.385163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:43.390488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb925-f578-4ea8-a9f9-8f254b821374 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:43.390910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb925-f578-4ea8-a9f9-8f254b821374 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:43.391061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb925-f578-4ea8-a9f9-8f254b821374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:43.457134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-111fb925-f578-4ea8-a9f9-8f254b821374 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:43.458616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-111fb925-f578-4ea8-a9f9-8f254b821374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:43.467538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:43.558607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:43.596293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f2e04-68b0-4c03-84dd-c66036d156f3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:43.597864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-241f2e04-68b0-4c03-84dd-c66036d156f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:43.626608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:43.702800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f2e04-68b0-4c03-84dd-c66036d156f3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:43.775720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:43.776367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:43.776650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:43.776967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:43.778621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:43.853766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:43.887339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:43.888804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe4e30-3190-49e5-b6f7-81f377e2b6e2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:43.908495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 745/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:45.969207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d64d0-b784-44ab-9f62-0b90d6465b4c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:45.969874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d64d0-b784-44ab-9f62-0b90d6465b4c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:45.969874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d64d0-b784-44ab-9f62-0b90d6465b4c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:45.971241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:45.976496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fb51-e1c7-4910-b9f5-30bbb380ccfd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:45.985665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fb51-e1c7-4910-b9f5-30bbb380ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:45.985665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fb51-e1c7-4910-b9f5-30bbb380ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:46.040467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c935fb51-e1c7-4910-b9f5-30bbb380ccfd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:46.041714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c935fb51-e1c7-4910-b9f5-30bbb380ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:46.141366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:46.180862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d64d0-b784-44ab-9f62-0b90d6465b4c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:46.182393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-497d64d0-b784-44ab-9f62-0b90d6465b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:46.288202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d64d0-b784-44ab-9f62-0b90d6465b4c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:46.328091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:46.366706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:46.367360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:46.367651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:46.367929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:46.369609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:46.478637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:46.480274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39189a-0bbe-4d9b-8184-488647166f0e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:46.499991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 718/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:48.534372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0fb3d-2e8a-4536-944b-025c3e973ecf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.534896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0fb3d-2e8a-4536-944b-025c3e973ecf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.534896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0fb3d-2e8a-4536-944b-025c3e973ecf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.536219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:48.541768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ec7c7-bcda-4ffc-879e-cd66e56fd6d2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.542194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ec7c7-bcda-4ffc-879e-cd66e56fd6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.542322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ec7c7-bcda-4ffc-879e-cd66e56fd6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.609554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-043ec7c7-bcda-4ffc-879e-cd66e56fd6d2 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:48.611466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-043ec7c7-bcda-4ffc-879e-cd66e56fd6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.629512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.709577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:48.747260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0fb3d-2e8a-4536-944b-025c3e973ecf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.748822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f2e0fb3d-2e8a-4536-944b-025c3e973ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.786673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.854187 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0fb3d-2e8a-4536-944b-025c3e973ecf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.928077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.928777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.929061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.929336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.930972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:49.011303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:49.037882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:49.039242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bc910-34e5-48a1-b656-840b484bde39 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:49.057999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 748/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:51.110166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecec86d-d4db-44ca-9801-f12ef0de1c44 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:51.110748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecec86d-d4db-44ca-9801-f12ef0de1c44 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:51.110913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecec86d-d4db-44ca-9801-f12ef0de1c44 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:51.112370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:36:51.117699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0b4b8-de9d-4aaa-a6e4-c7aa22826826 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:51.118111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0b4b8-de9d-4aaa-a6e4-c7aa22826826 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:51.118259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0b4b8-de9d-4aaa-a6e4-c7aa22826826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:51.182879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e3e0b4b8-de9d-4aaa-a6e4-c7aa22826826 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:51.184099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e3e0b4b8-de9d-4aaa-a6e4-c7aa22826826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:51.281865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:51.319230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecec86d-d4db-44ca-9801-f12ef0de1c44 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:51.320807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2ecec86d-d4db-44ca-9801-f12ef0de1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:51.423388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecec86d-d4db-44ca-9801-f12ef0de1c44 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:51.470787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:51.497025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:51.497669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:51.497970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:51.498246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:51.499894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:51.612449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:51.613942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ef93e-b3e2-48df-b7cc-ba6d05ffe753 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:51.632530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 721/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:01.279983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:01.630268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=66 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:11.419239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c937-6162-4bd7-8ecd-f52cd738ab11 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:11.419836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c937-6162-4bd7-8ecd-f52cd738ab11 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:11.419836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c937-6162-4bd7-8ecd-f52cd738ab11 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.421243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 750/1471] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:11.427160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e47b79-4da7-4431-93f0-1a3513c3dc46 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:11.427623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e47b79-4da7-4431-93f0-1a3513c3dc46 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:11.427729 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e47b79-4da7-4431-93f0-1a3513c3dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.493648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-46e47b79-4da7-4431-93f0-1a3513c3dc46 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:11.494848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-46e47b79-4da7-4431-93f0-1a3513c3dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:11.589896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 722/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:11.638692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d337e3-c17d-430b-a811-b1516f80eac1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:11.638986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d337e3-c17d-430b-a811-b1516f80eac1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:11.639271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d337e3-c17d-430b-a811-b1516f80eac1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.640730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 751/1473] 10.222.0.47 () {54 vars in 961 bytes} [Mon Jul 18 10:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:11.645511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d26ef-869e-4e44-a527-83012083fc08 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:11.645810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d26ef-869e-4e44-a527-83012083fc08 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:11.646101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d26ef-869e-4e44-a527-83012083fc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.723939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b06d26ef-869e-4e44-a527-83012083fc08 None None] MFA Rules not processed for user `aefb3626f45544ba81603a722821fce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:11.725195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b06d26ef-869e-4e44-a527-83012083fc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:11.745893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:11.804331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 723/1474] 10.222.0.47 () {58 vars in 1058 bytes} [Mon Jul 18 10:37:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:11.811768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d337e3-c17d-430b-a811-b1516f80eac1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:11.813939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-48d337e3-c17d-430b-a811-b1516f80eac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:11.916554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d337e3-c17d-430b-a811-b1516f80eac1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:11.988960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:11.989611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:11.989898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:11.990180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:11.991830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:12.047922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:12.104364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:12.105835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d5439-54e1-4779-a6b8-98af86f2200a service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:12.124227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 752/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:14.258899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf8936-23bd-4214-aec8-b25803c81fc4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:14.259467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf8936-23bd-4214-aec8-b25803c81fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:14.259467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf8936-23bd-4214-aec8-b25803c81fc4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:14.260895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 724/1476] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:14.267349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252336-aec5-4ed7-af3d-21a4f4095684 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:14.267349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252336-aec5-4ed7-af3d-21a4f4095684 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:14.267349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252336-aec5-4ed7-af3d-21a4f4095684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:14.331112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d4252336-aec5-4ed7-af3d-21a4f4095684 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:14.332316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d4252336-aec5-4ed7-af3d-21a4f4095684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:14.428184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 753/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:14.451346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf8936-23bd-4214-aec8-b25803c81fc4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:14.453436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a5cf8936-23bd-4214-aec8-b25803c81fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:14.581695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf8936-23bd-4214-aec8-b25803c81fc4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:14.630714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:14.681219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:14.681874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:14.682183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:14.682459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:14.683189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:14.684990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:37:14.684990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:37:14.684990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:37:14.686132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f895e0a-5f8a-4712-a35e-286c9669b757 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:14.704894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 725/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Jul 18 10:37:14 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:37:16.771526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d9bb-656c-4936-a548-aac1bc1ca312 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:16.772095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d9bb-656c-4936-a548-aac1bc1ca312 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:16.772095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d9bb-656c-4936-a548-aac1bc1ca312 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:16.773563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 754/1479] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:16.778935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eb3c5-da24-40c3-8828-eedd386ffa56 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:16.779349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eb3c5-da24-40c3-8828-eedd386ffa56 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:16.779525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eb3c5-da24-40c3-8828-eedd386ffa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:16.845279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-866eb3c5-da24-40c3-8828-eedd386ffa56 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:16.846509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-866eb3c5-da24-40c3-8828-eedd386ffa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:16.942723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 726/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:16.989721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147d9bb-656c-4936-a548-aac1bc1ca312 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:16.989721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f147d9bb-656c-4936-a548-aac1bc1ca312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:16.998081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:17.133053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147d9bb-656c-4936-a548-aac1bc1ca312 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:17.213012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:17.246455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:17.247110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:17.247398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:17.247677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:17.249387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:17.338769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:17.340348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59019cb-ef6e-49c8-8e11-1fdfed09bf95 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:17.357886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 755/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:19.551748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1cad-e0c4-4496-81b0-3ced3508faa6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:19.552359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1cad-e0c4-4496-81b0-3ced3508faa6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:19.552359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1cad-e0c4-4496-81b0-3ced3508faa6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:19.553643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 727/1482] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:19.560644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f071e9-b8af-4423-a183-1f501a36dbeb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:19.561142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f071e9-b8af-4423-a183-1f501a36dbeb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:19.561353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f071e9-b8af-4423-a183-1f501a36dbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:19.626431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-18f071e9-b8af-4423-a183-1f501a36dbeb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:19.627632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-18f071e9-b8af-4423-a183-1f501a36dbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:19.672242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:19.722891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 756/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:19.760176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1cad-e0c4-4496-81b0-3ced3508faa6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:19.761808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-adad1cad-e0c4-4496-81b0-3ced3508faa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:19.785106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:19.866254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1cad-e0c4-4496-81b0-3ced3508faa6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:19.938345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:19.938990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:19.939274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:19.939551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:19.941263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:20.016423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:20.051545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:20.052999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323ef30-3abd-42d4-825d-49a2fc011289 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:20.071505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 728/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:22.161047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3e0f-db2c-4199-9be2-463acc049a60 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:22.161531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3e0f-db2c-4199-9be2-463acc049a60 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:22.161640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3e0f-db2c-4199-9be2-463acc049a60 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:22.163057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 757/1485] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:22.168481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb373a-5681-415d-8ec8-fc6a5c3a42ad None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:22.168919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb373a-5681-415d-8ec8-fc6a5c3a42ad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:22.169073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb373a-5681-415d-8ec8-fc6a5c3a42ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:22.238281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d1bb373a-5681-415d-8ec8-fc6a5c3a42ad None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:22.239505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d1bb373a-5681-415d-8ec8-fc6a5c3a42ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:22.335867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 729/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:22.373456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd3e0f-db2c-4199-9be2-463acc049a60 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:22.376140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4afd3e0f-db2c-4199-9be2-463acc049a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:22.480301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd3e0f-db2c-4199-9be2-463acc049a60 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:22.552354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:22.554203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:22.554859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:22.555144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:22.555423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:22.557114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:22.666362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:22.667801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee85111-f9c9-4950-a713-e1911154b9c2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:22.685876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 758/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:24.896040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552715c1-d601-4448-8d95-d5d9ef160368 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:24.896735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552715c1-d601-4448-8d95-d5d9ef160368 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:24.896735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552715c1-d601-4448-8d95-d5d9ef160368 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:24.898017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 730/1488] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:24.903283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3395485-9b1e-4a80-ba4d-496281eed17f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:24.903738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3395485-9b1e-4a80-ba4d-496281eed17f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:24.903848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3395485-9b1e-4a80-ba4d-496281eed17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:24.968084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a3395485-9b1e-4a80-ba4d-496281eed17f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:24.969331 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a3395485-9b1e-4a80-ba4d-496281eed17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:25.021491 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:25.065446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 759/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:25.105639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552715c1-d601-4448-8d95-d5d9ef160368 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:25.108098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-552715c1-d601-4448-8d95-d5d9ef160368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:25.146363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:25.215751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552715c1-d601-4448-8d95-d5d9ef160368 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:25.290706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:25.291351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:25.291677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:25.291955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:25.293621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:25.378284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:25.408456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:25.409832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca2b55-b054-4761-b99d-21fb62a894eb service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:25.427907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 731/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:27.931960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa1bd-9a03-4bf5-9785-3da98db5f694 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.932569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa1bd-9a03-4bf5-9785-3da98db5f694 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.932569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa1bd-9a03-4bf5-9785-3da98db5f694 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.933946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 760/1491] 10.222.0.47 () {54 vars in 926 bytes} [Mon Jul 18 10:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:37:27.940003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a0cab-7a34-4b8f-844a-6b177e2ea35a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.940434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a0cab-7a34-4b8f-844a-6b177e2ea35a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.940566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a0cab-7a34-4b8f-844a-6b177e2ea35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.123242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1e7a0cab-7a34-4b8f-844a-6b177e2ea35a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:28.125124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1e7a0cab-7a34-4b8f-844a-6b177e2ea35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.220357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 732/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:28.260444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa1bd-9a03-4bf5-9785-3da98db5f694 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.262049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aaafa1bd-9a03-4bf5-9785-3da98db5f694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.365053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa1bd-9a03-4bf5-9785-3da98db5f694 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.438196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.438836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.439124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.439399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.441064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.449620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:28.550015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.551477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c6a91-0ef6-4058-a8fe-3cf02a527f3b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.570938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 761/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:38.217818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=82 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:38.568635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=50 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:05.142224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.142830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.142830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.216276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:05.217492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:05.312517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 733/1494] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:05.320837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.321306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.321401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.395860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:05.397275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:05.493623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 762/1495] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:05.498619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:05.499843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:05.511699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe4ec2-6465-45a4-bffe-e22a6d9acc9b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:05.573241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:05.608608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:05.609344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.609916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.610424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.611507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:39:05.611818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:05.612892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:05.612892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:39:05.612892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:39:05.613782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:05.627095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 734/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:39:05.635867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:05.645519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbb49a-b806-4f58-9aa5-4a58e9f410a5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:05.708226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:05.742020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:05.742680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.742965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.743243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.743949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:05.746216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:05.746216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:39:05.746216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:39:05.747467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:05.863486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 763/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:39:05 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:05.867633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:05.869154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:05.903696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd40b01-f239-4fe4-a489-18de9ca59da7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:05.996301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:05.996908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:05.997192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:05.997464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:05.998403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:05.999413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:05.999413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:39:05.999413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:39:06.000608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:06.131176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 735/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:39:05 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:06.136037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:06.137657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.198952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05913c6b-fb97-4306-a092-9cc49b51ac9d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:06.283479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:06.294485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:39:06.300149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:39:06.300761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:06.311400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 764/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:06.315540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:06.317087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.328652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872570ca-0d3e-4d92-a4a2-fb35fa3622a7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:06.353655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:06.424132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:06.424812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.425094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.425367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.426103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:06.427214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:06.427214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:39:06.427214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:39:06.428085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71583f8-1819-44f6-a402-46a37251b16a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:06.440386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 736/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:06.444396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:06.445972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.484490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b56e-a5d5-4623-b9d3-206ddf7a634b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:06.590520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:06.591355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.591652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.591976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] PATH_INFO: `/v3/projects/955c97a7c89945ba9db7b953a8e337fc/users/684e647bfbe846d884006540fbb32150/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.614986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955c97a7c89945ba9db7b953a8e337fc, user_id=684e647bfbe846d884006540fbb32150, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:06.616016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:06.616016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:39:06.616016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:39:06.617994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9b877-864d-4a08-83ed-1039134c77c1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:06.749479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 765/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:39:06 2022] PUT /identity/v3/projects/955c97a7c89945ba9db7b953a8e337fc/users/684e647bfbe846d884006540fbb32150/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:39:06.762036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef6ab-19da-45ec-a0ff-3e368f3c3046 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.762332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef6ab-19da-45ec-a0ff-3e368f3c3046 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.762620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef6ab-19da-45ec-a0ff-3e368f3c3046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.819641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e73ef6ab-19da-45ec-a0ff-3e368f3c3046 None None] MFA Rules not processed for user `684e647bfbe846d884006540fbb32150`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:06.820904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e73ef6ab-19da-45ec-a0ff-3e368f3c3046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:06.900490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 737/1502] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:06.914573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8d22d-7ebb-4263-8ff5-74dce21a6f63 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.914874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8d22d-7ebb-4263-8ff5-74dce21a6f63 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.915160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8d22d-7ebb-4263-8ff5-74dce21a6f63 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.916592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 766/1503] 10.222.0.47 () {54 vars in 964 bytes} [Mon Jul 18 10:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:06.921616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8695d-0227-4aaf-8642-b92cb64767a1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:06.921933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8695d-0227-4aaf-8642-b92cb64767a1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:06.922215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8695d-0227-4aaf-8642-b92cb64767a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:06.951793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:06.996659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-81e8695d-0227-4aaf-8642-b92cb64767a1 None None] MFA Rules not processed for user `bd8f56a4b2d94ec1aec6f0d044cbc1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:06.997904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-81e8695d-0227-4aaf-8642-b92cb64767a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:07.075162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 738/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Mon Jul 18 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:07.083520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8d22d-7ebb-4263-8ff5-74dce21a6f63 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:07.085188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fe8d22d-7ebb-4263-8ff5-74dce21a6f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:07.139305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:07.200534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8d22d-7ebb-4263-8ff5-74dce21a6f63 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:07.276312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:07.276999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:07.277288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:07.277564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:07.279209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:07.376461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:07.377935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:07.395462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 767/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:07.438402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c17e0-e171-40db-b2a7-ba2804b2cc11 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:07.438836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c17e0-e171-40db-b2a7-ba2804b2cc11 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:07.442518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c17e0-e171-40db-b2a7-ba2804b2cc11 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:07.442518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 739/1506] 10.222.0.47 () {52 vars in 829 bytes} [Mon Jul 18 10:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:39:07.443825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:07.445077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:07.481133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:07.564301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8370b8-dbac-4450-9f92-ba4d4ca94160 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:07.657192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:07.658062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:07.658345 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:07.658621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] PATH_INFO: `/v3/users/684e647bfbe846d884006540fbb32150` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:07.671371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=684e647bfbe846d884006540fbb32150)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:07.672399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:07.672399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:39:07.672399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:39:07.673657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:07.683960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:07.925265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684e647bfbe846d884006540fbb32150'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:07.937532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684e647bfbe846d884006540fbb32150'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:08.044651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684e647bfbe846d884006540fbb32150'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:08.045236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684e647bfbe846d884006540fbb32150'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:08.045668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684e647bfbe846d884006540fbb32150'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:08.069501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 768/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:07 2022] DELETE /identity/v3/users/684e647bfbe846d884006540fbb32150 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:08.074100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:08.074396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:08.074683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:08.130799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:08.132031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:08.214837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:08.228138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 740/1508] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:08.238895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:08.240135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:08.350550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456acbab-b52b-44de-8f51-106e2852eb7c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:08.406634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:08.426584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:08.427245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:08.427532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:08.427813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:08.429515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:08.550210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:08.551662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195843-7020-4286-b186-93ebae746460 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:08.571476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 769/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:09.160879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:09.162111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:09.185863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20233e6-1b4b-460c-9a04-145f703129cf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:09.286118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:09.286976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:09.287258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:09.287534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] PATH_INFO: `/v3/projects/955c97a7c89945ba9db7b953a8e337fc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:09.298116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] RBAC: Authorizing `identity:delete_project(project_id=955c97a7c89945ba9db7b953a8e337fc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:09.300011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:39:09.300011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:39:09.300011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:39:09.300318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:09.421082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955c97a7c89945ba9db7b953a8e337fc', 'request_id': 'req-ea3bda63-d9c8-450f-9c22-03e0463a640c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:09.524847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955c97a7c89945ba9db7b953a8e337fc', 'request_id': 'req-ea3bda63-d9c8-450f-9c22-03e0463a640c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:09.532268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955c97a7c89945ba9db7b953a8e337fc', 'request_id': 'req-ea3bda63-d9c8-450f-9c22-03e0463a640c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:09.564996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] The token cache is being invalidate because project 955c97a7c89945ba9db7b953a8e337fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:09.565287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:09.565839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea3bda63-d9c8-450f-9c22-03e0463a640c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:09.567188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 741/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:09 2022] DELETE /identity/v3/projects/955c97a7c89945ba9db7b953a8e337fc => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:18.569852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=62 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:19.562830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:57.466845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffc47c-d5f2-4b3f-9970-171c9cc9ee8c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:57.467399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffc47c-d5f2-4b3f-9970-171c9cc9ee8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:57.467399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffc47c-d5f2-4b3f-9970-171c9cc9ee8c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:57.468770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:57.473841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5924af-4b57-4ae7-80ee-950380308b28 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:57.474304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5924af-4b57-4ae7-80ee-950380308b28 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:57.474406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5924af-4b57-4ae7-80ee-950380308b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:57.560525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3e5924af-4b57-4ae7-80ee-950380308b28 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:57.561784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3e5924af-4b57-4ae7-80ee-950380308b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:57.664655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:40:57.819026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffc47c-d5f2-4b3f-9970-171c9cc9ee8c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:57.821705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-05ffc47c-d5f2-4b3f-9970-171c9cc9ee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:57.939056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffc47c-d5f2-4b3f-9970-171c9cc9ee8c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:57.944508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63743f-ddef-40a0-ac3c-ec789a80253d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:57.945178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63743f-ddef-40a0-ac3c-ec789a80253d service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:57.945467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63743f-ddef-40a0-ac3c-ec789a80253d service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:57.945743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63743f-ddef-40a0-ac3c-ec789a80253d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:57.954607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63743f-ddef-40a0-ac3c-ec789a80253d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:57.956457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63743f-ddef-40a0-ac3c-ec789a80253d service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:57.966928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 771/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:40:58.813345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36d3d-1fa8-43f8-8a9e-cac1b900a4de None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:58.813886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36d3d-1fa8-43f8-8a9e-cac1b900a4de None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:58.813886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36d3d-1fa8-43f8-8a9e-cac1b900a4de None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:58.815221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 743/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:40:58.820016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506537f-e1cb-4533-9d66-6ddf12279d58 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:58.820460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506537f-e1cb-4533-9d66-6ddf12279d58 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:58.820574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506537f-e1cb-4533-9d66-6ddf12279d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:58.842161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1506537f-e1cb-4533-9d66-6ddf12279d58 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:58.843364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1506537f-e1cb-4533-9d66-6ddf12279d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:58.851386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 772/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:59.005223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b36d3d-1fa8-43f8-8a9e-cac1b900a4de None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:59.021534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b36d3d-1fa8-43f8-8a9e-cac1b900a4de None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:59.030016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332a632-17f7-4fc4-9216-d56e0629a65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:59.030671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332a632-17f7-4fc4-9216-d56e0629a65b service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:59.030983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332a632-17f7-4fc4-9216-d56e0629a65b service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:59.031262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332a632-17f7-4fc4-9216-d56e0629a65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:59.039893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332a632-17f7-4fc4-9216-d56e0629a65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:59.041547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332a632-17f7-4fc4-9216-d56e0629a65b service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:59.050899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 744/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:59.102449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24903680-fffd-4dc4-9c9b-d817575294d3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:59.102768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24903680-fffd-4dc4-9c9b-d817575294d3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:59.103057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24903680-fffd-4dc4-9c9b-d817575294d3 None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:59.104385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 773/1517] 10.222.0.59 () {54 vars in 925 bytes} [Mon Jul 18 10:40:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:40:59.111113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878170a4-2d68-4498-9f7f-69fe76017ff7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:59.116428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878170a4-2d68-4498-9f7f-69fe76017ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:59.116428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878170a4-2d68-4498-9f7f-69fe76017ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:59.175070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-878170a4-2d68-4498-9f7f-69fe76017ff7 None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:59.176286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-878170a4-2d68-4498-9f7f-69fe76017ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:59.194147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:59.250304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 745/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Jul 18 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:03.440440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7543aee-0bb7-4ce4-a6e0-98f64d58e27f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:03.441051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7543aee-0bb7-4ce4-a6e0-98f64d58e27f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:03.441051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7543aee-0bb7-4ce4-a6e0-98f64d58e27f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:03.442301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 774/1519] 10.222.0.47 () {54 vars in 961 bytes} [Mon Jul 18 10:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:03.448017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd724ff-1168-47c4-a7a7-4fbd71461d80 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:03.448483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd724ff-1168-47c4-a7a7-4fbd71461d80 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:03.448730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd724ff-1168-47c4-a7a7-4fbd71461d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:03.466984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ffd724ff-1168-47c4-a7a7-4fbd71461d80 None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:03.468182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ffd724ff-1168-47c4-a7a7-4fbd71461d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:03.476664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 746/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Mon Jul 18 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:03.484510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7543aee-0bb7-4ce4-a6e0-98f64d58e27f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:03.498104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7543aee-0bb7-4ce4-a6e0-98f64d58e27f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:03.503012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b978-1f9a-459b-afbc-64e6ab16a517 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:03.503686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b978-1f9a-459b-afbc-64e6ab16a517 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:03.503969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b978-1f9a-459b-afbc-64e6ab16a517 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:03.504243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b978-1f9a-459b-afbc-64e6ab16a517 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:03.513273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b978-1f9a-459b-afbc-64e6ab16a517 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:03.514849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b978-1f9a-459b-afbc-64e6ab16a517 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:03.524482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 775/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:10.286840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796c769-7380-4a30-9935-bc3d22bf7d96 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:10.287296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796c769-7380-4a30-9935-bc3d22bf7d96 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:10.287405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796c769-7380-4a30-9935-bc3d22bf7d96 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:10.288808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 747/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:10.295235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6034cf1-f24b-455f-a7c4-53392fd6b10f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:10.295644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6034cf1-f24b-455f-a7c4-53392fd6b10f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:10.295806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6034cf1-f24b-455f-a7c4-53392fd6b10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:10.313854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b6034cf1-f24b-455f-a7c4-53392fd6b10f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:10.315065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b6034cf1-f24b-455f-a7c4-53392fd6b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:10.323182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 776/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:10.477590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796c769-7380-4a30-9935-bc3d22bf7d96 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:10.491704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796c769-7380-4a30-9935-bc3d22bf7d96 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:10.497050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17048a2-f7f2-4747-886b-a3e69d37c4ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:10.497717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17048a2-f7f2-4747-886b-a3e69d37c4ed service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:10.497997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17048a2-f7f2-4747-886b-a3e69d37c4ed service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:10.498274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17048a2-f7f2-4747-886b-a3e69d37c4ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:10.506603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17048a2-f7f2-4747-886b-a3e69d37c4ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:10.508901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17048a2-f7f2-4747-886b-a3e69d37c4ed service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:10.518499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 748/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:11.363139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fac88-5294-4e86-af42-6ffe82718fe5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:11.363717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fac88-5294-4e86-af42-6ffe82718fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:11.363717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fac88-5294-4e86-af42-6ffe82718fe5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:11.365125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:11.370495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a006-6f77-48a5-a7ea-026d580dd756 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:11.370965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a006-6f77-48a5-a7ea-026d580dd756 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:11.371071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a006-6f77-48a5-a7ea-026d580dd756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:11.390736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f799a006-6f77-48a5-a7ea-026d580dd756 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:11.391972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f799a006-6f77-48a5-a7ea-026d580dd756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:11.404032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:11.559060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fac88-5294-4e86-af42-6ffe82718fe5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:11.575387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fac88-5294-4e86-af42-6ffe82718fe5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:11.580880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c28a5-88bc-4acb-bbfd-8722488e73cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:11.581797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c28a5-88bc-4acb-bbfd-8722488e73cb service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:11.582085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c28a5-88bc-4acb-bbfd-8722488e73cb service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:11.582549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c28a5-88bc-4acb-bbfd-8722488e73cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:11.593747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c28a5-88bc-4acb-bbfd-8722488e73cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:11.595934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c28a5-88bc-4acb-bbfd-8722488e73cb service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:11.608575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 778/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:15.489196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53c815-2113-4aae-a713-9e63675a5a62 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:15.489762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53c815-2113-4aae-a713-9e63675a5a62 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:15.489762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53c815-2113-4aae-a713-9e63675a5a62 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:15.491142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:15.496421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ef91b-d007-499e-9e30-914bc0d5c278 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:15.496854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ef91b-d007-499e-9e30-914bc0d5c278 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:15.497017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ef91b-d007-499e-9e30-914bc0d5c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:15.514863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-415ef91b-d007-499e-9e30-914bc0d5c278 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:15.516070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-415ef91b-d007-499e-9e30-914bc0d5c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:15.524934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:15.563173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53c815-2113-4aae-a713-9e63675a5a62 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:15.564905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ce53c815-2113-4aae-a713-9e63675a5a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:15.644920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53c815-2113-4aae-a713-9e63675a5a62 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:15.649097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be45f7-4eca-4c40-88be-31209a3c8395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:15.649743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be45f7-4eca-4c40-88be-31209a3c8395 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:15.650025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be45f7-4eca-4c40-88be-31209a3c8395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:15.650302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be45f7-4eca-4c40-88be-31209a3c8395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:15.659274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be45f7-4eca-4c40-88be-31209a3c8395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:15.660727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be45f7-4eca-4c40-88be-31209a3c8395 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:15.670862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 751/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:17.675384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39350447-6a28-4ef5-a8e0-5bb76c1427f3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:17.675926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39350447-6a28-4ef5-a8e0-5bb76c1427f3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:17.675926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39350447-6a28-4ef5-a8e0-5bb76c1427f3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:17.677363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:17.683007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70765a-a8ed-4133-9357-7e43a73d3360 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:17.683446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70765a-a8ed-4133-9357-7e43a73d3360 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:17.683545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70765a-a8ed-4133-9357-7e43a73d3360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:17.701374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4b70765a-a8ed-4133-9357-7e43a73d3360 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:17.702551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4b70765a-a8ed-4133-9357-7e43a73d3360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:17.710521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:17.746507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39350447-6a28-4ef5-a8e0-5bb76c1427f3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:17.757130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39350447-6a28-4ef5-a8e0-5bb76c1427f3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:17.761642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b489197-d2c9-4a0d-9685-98750dacd931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:17.762288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b489197-d2c9-4a0d-9685-98750dacd931 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:17.762579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b489197-d2c9-4a0d-9685-98750dacd931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:17.762886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b489197-d2c9-4a0d-9685-98750dacd931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:17.771916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b489197-d2c9-4a0d-9685-98750dacd931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:17.773453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b489197-d2c9-4a0d-9685-98750dacd931 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:17.783022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 781/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:21.340766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65274c-b672-48a9-91c6-fb4e489a991f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:21.341314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65274c-b672-48a9-91c6-fb4e489a991f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:21.341314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65274c-b672-48a9-91c6-fb4e489a991f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:21.342706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:21.347810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45b822-07d9-4417-8d81-79c7dffc329c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:21.348236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45b822-07d9-4417-8d81-79c7dffc329c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:21.348386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45b822-07d9-4417-8d81-79c7dffc329c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:21.355666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:41:21.368364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3f45b822-07d9-4417-8d81-79c7dffc329c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:21.370625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3f45b822-07d9-4417-8d81-79c7dffc329c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:21.377539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:21.685617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65274c-b672-48a9-91c6-fb4e489a991f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:21.687305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2a65274c-b672-48a9-91c6-fb4e489a991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:21.753256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:41:21.766775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65274c-b672-48a9-91c6-fb4e489a991f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:21.770367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996c78-b4b5-4028-a3f7-aa072533299e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:21.771011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996c78-b4b5-4028-a3f7-aa072533299e service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:21.771296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996c78-b4b5-4028-a3f7-aa072533299e service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:21.771579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996c78-b4b5-4028-a3f7-aa072533299e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:21.781500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44996c78-b4b5-4028-a3f7-aa072533299e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:21.791901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44996c78-b4b5-4028-a3f7-aa072533299e service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:21.792457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 754/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:23.832459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1edb3-3849-455c-a06b-73470d3bb3b3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:23.833047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1edb3-3849-455c-a06b-73470d3bb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:23.833047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1edb3-3849-455c-a06b-73470d3bb3b3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:23.834487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:41:23.840108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d715ccd-d74a-4896-b996-0763d4ca6cc4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:23.840535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d715ccd-d74a-4896-b996-0763d4ca6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:23.840668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d715ccd-d74a-4896-b996-0763d4ca6cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:23.858114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1d715ccd-d74a-4896-b996-0763d4ca6cc4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:41:23.859308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1d715ccd-d74a-4896-b996-0763d4ca6cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:23.867757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:41:24.175829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1edb3-3849-455c-a06b-73470d3bb3b3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:41:24.177600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dbf1edb3-3849-455c-a06b-73470d3bb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:41:24.191236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1edb3-3849-455c-a06b-73470d3bb3b3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:41:24.194744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d76dbb-f983-4ec6-9d44-244874ac9a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:41:24.195457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d76dbb-f983-4ec6-9d44-244874ac9a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:41:24.195752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d76dbb-f983-4ec6-9d44-244874ac9a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:41:24.196029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d76dbb-f983-4ec6-9d44-244874ac9a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:41:24.204931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d76dbb-f983-4ec6-9d44-244874ac9a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:41:24.206617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d76dbb-f983-4ec6-9d44-244874ac9a2b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:41:24.215958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 784/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:41:33.853363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=11 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:41:34.211898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:53.670947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:53.678835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:53.678835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:53.693957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:53.695810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:53.708616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 756/1540] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:53.718759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:53.719409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:53.719409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:53.748872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:53.750158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:53.767484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 785/1541] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:53.772159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:53.786417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2efc4-8e1b-470f-866d-377898b799c9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:53.797073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:53.797776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:53.798063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:53.798342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:53.799117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:53.799431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:53.801154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625566-698b-4c61-b7d1-97a50e37c53a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:53.816407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 757/1542] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:53.824534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:53.839319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acb928-9abb-4c1f-b69b-3fed01c80311 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:53.849115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413160c9-c564-4613-a7c7-f628953eef71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:53.849803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413160c9-c564-4613-a7c7-f628953eef71 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:53.850103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413160c9-c564-4613-a7c7-f628953eef71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:53.850377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413160c9-c564-4613-a7c7-f628953eef71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:53.851812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413160c9-c564-4613-a7c7-f628953eef71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:53.854248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413160c9-c564-4613-a7c7-f628953eef71 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:53.956762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a10c-d640-4350-a6bd-6ebc90ad8155 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:53.956960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a10c-d640-4350-a6bd-6ebc90ad8155 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:53.957253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a10c-d640-4350-a6bd-6ebc90ad8155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:53.989973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-35e5a10c-d640-4350-a6bd-6ebc90ad8155 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:53.992119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-35e5a10c-d640-4350-a6bd-6ebc90ad8155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.000837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 758/1543] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 18 10:42:54.009504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.009844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.010073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.033376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.034593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.048600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 759/1544] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:42:54.053040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.063681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdc3a3-b16a-4ab6-8652-35e4378542b0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.074479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.075969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.075969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.075969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.076483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.076834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.078560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006b61-c04c-4377-8c83-273d42be297a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.092983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 760/1545] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.094377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.094674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.094959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.114700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.115912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.131055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 761/1546] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.131055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.138763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 786/1547] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:53 2022] POST /identity/v3/projects => generated 371 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:54.138763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb2647-db9c-4f8d-b2ac-56c0ba9c402d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.138763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb2647-db9c-4f8d-b2ac-56c0ba9c402d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.138763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb2647-db9c-4f8d-b2ac-56c0ba9c402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.144295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dd869-30ef-434e-b4cb-a89018cc197a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.177719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e6fb2647-db9c-4f8d-b2ac-56c0ba9c402d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.179901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e6fb2647-db9c-4f8d-b2ac-56c0ba9c402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.206917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.207513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.207802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.208085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.208816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.210378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:54.210378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:42:54.210378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:42:54.211608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.274685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 787/1548] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.276095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68343082-7c1b-4051-938d-6b537e7dd455 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.276237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68343082-7c1b-4051-938d-6b537e7dd455 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.276763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68343082-7c1b-4051-938d-6b537e7dd455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.297031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-68343082-7c1b-4051-938d-6b537e7dd455 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.298251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-68343082-7c1b-4051-938d-6b537e7dd455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.307551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 788/1549] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.309064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09032bc-03d2-43e6-8787-e4ca4d07530e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.309356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09032bc-03d2-43e6-8787-e4ca4d07530e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.309645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09032bc-03d2-43e6-8787-e4ca4d07530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.311337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 762/1550] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:54.312568 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.328287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e09032bc-03d2-43e6-8787-e4ca4d07530e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.329534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e09032bc-03d2-43e6-8787-e4ca4d07530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.377114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52b6f-27fd-4a87-b2b0-1a0b2306f104 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.381953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.382628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.382910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.383188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.383910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.385869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.389722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 789/1551] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.391003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bffe8a-2322-49d8-92ad-94fde71df179 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.391300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bffe8a-2322-49d8-92ad-94fde71df179 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.391580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bffe8a-2322-49d8-92ad-94fde71df179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.411485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-94bffe8a-2322-49d8-92ad-94fde71df179 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.417697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94bffe8a-2322-49d8-92ad-94fde71df179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.421482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 790/1552] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.422669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358bce4-dc71-4797-9e43-63564e83c2cd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.422959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358bce4-dc71-4797-9e43-63564e83c2cd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.423242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358bce4-dc71-4797-9e43-63564e83c2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.440422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e358bce4-dc71-4797-9e43-63564e83c2cd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.441637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e358bce4-dc71-4797-9e43-63564e83c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.449584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 791/1553] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.450749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.451041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.451333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.469134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.470222 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.479149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 792/1554] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.479915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.489896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9995c-5d22-4ec0-aaa2-9027afadb666 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.494745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 763/1555] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:54.494745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.494954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.495597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.495890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.496170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.496949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.497266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.498296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:54.498296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:42:54.498296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:42:54.499721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afac652-4f18-4acc-abb7-990a73f1270d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.503019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119ab15-9d5c-4a34-915d-123d52595fa7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.507728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfbe45-aecd-4524-9a92-c326b17f7cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.508383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbe45-aecd-4524-9a92-c326b17f7cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.508671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbe45-aecd-4524-9a92-c326b17f7cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.508987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbe45-aecd-4524-9a92-c326b17f7cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.509950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfbe45-aecd-4524-9a92-c326b17f7cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.518787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 793/1556] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.518975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfbe45-aecd-4524-9a92-c326b17f7cc5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.519081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9e3b1-62e7-4608-914c-6bc1be7f6734 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.519081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9e3b1-62e7-4608-914c-6bc1be7f6734 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.519081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9e3b1-62e7-4608-914c-6bc1be7f6734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.532637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06e9e3b1-62e7-4608-914c-6bc1be7f6734 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.534010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06e9e3b1-62e7-4608-914c-6bc1be7f6734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.542622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 794/1557] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.543892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346b36-9ce2-4d47-8d9b-6600f8a922d7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.544182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346b36-9ce2-4d47-8d9b-6600f8a922d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.544481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346b36-9ce2-4d47-8d9b-6600f8a922d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.564500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-07346b36-9ce2-4d47-8d9b-6600f8a922d7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.565748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07346b36-9ce2-4d47-8d9b-6600f8a922d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.573937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 795/1558] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.575319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.575609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.575891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.593969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.595121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.603556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 796/1559] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.604302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.609991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:54.614399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658664ca-8aa8-4fc2-a37f-a30034b391e5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.619099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 764/1560] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:54.619099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.619099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.619099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.622195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.622850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.623140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.623421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.624156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.624468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.625499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:54.626151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.636411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.637658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.639013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 797/1561] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.639831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.652260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 765/1562] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.653280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.654927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac096e5f-7abe-4252-8587-2c41815f2120 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.659693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c9ec7-6573-4737-962b-73ad06d1b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.664149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c9ec7-6573-4737-962b-73ad06d1b53b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.664149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c9ec7-6573-4737-962b-73ad06d1b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.664149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c9ec7-6573-4737-962b-73ad06d1b53b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.664149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c9ec7-6573-4737-962b-73ad06d1b53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.664501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778b0d5-559c-4c01-ba27-04786716864f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.664582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c9ec7-6573-4737-962b-73ad06d1b53b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.667303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.667942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.668227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.668504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.669220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.671097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.673933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 798/1563] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:54.675159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.675483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.675789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.694117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:54.695844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:54.703913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 799/1564] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:54.704708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.714830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a05e-44a8-4637-b6e7-b09592b5ac0d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.721518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.721518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.721518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.721518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.722530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.722530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.723204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.734798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 800/1565] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.735570 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.744675 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f0f2d-a4a8-4dd4-a0bf-2255f642d346 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.749382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.750021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.750310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.750588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.751312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.751621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.753242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.765540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 801/1566] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.766320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.776306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2c75e-c16a-4807-8b1f-ce83207b9154 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.781078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.782096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.782096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.782367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.783015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.784597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.785237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 766/1567] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:54.786039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.795111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 802/1568] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:54.796171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.806923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519c471-af4f-405e-ac8d-ea744aa79f34 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.849077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc422bfc-12af-43b9-be8c-5ef3e97de95b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.853849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.854513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.854803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.855121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.855846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.858841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.865153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.865810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.866125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.866629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.867360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.867668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.869353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.888580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 803/1569] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.888580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.891521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef697f-3273-4d67-b04f-2ed3acd81621 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.896104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.896779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.897068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.897421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.898589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.900162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.910358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 804/1570] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:54.911129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.920220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eb7f8-8fe1-40b5-95bf-4aa63c40d930 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:54.924868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:54.925486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:54.925776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:54.926057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:54.926772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:42:54.927081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:54.928631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:54.932575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 767/1571] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:54.933912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.941492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 805/1572] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:42:54.942289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:54.952385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76517596-58fe-43ab-af90-8b20a5d0d0f8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.006408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e7f30-1bdf-4a44-b5b5-1defe03aa3da admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.012105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.012826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.013120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.013439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.014094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.014341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.015055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.015352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.015665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.016132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.016622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.032983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.032983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:55.138953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 768/1573] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/projects => generated 393 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.139774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.172520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 806/1574] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:54 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.173337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.187739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:55.202610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db37ac-b7ce-48c0-ad0e-73c77dfcebca admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.237031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a55a-d9e8-4865-b59a-8e31adfe49d0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.242206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.244888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.244888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.244888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.244888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.245741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.255796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 807/1575] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:55.258172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.262758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.263734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.264025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.264308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.265048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.267907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.269113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba23ee-b3ae-414b-9855-b51f1321eb01 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.274213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.274875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.275164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.275443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.276128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.278051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.394353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 769/1576] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.394353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.402332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f117a34-d73c-4810-9d73-cd1fcf7769b2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.405212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 808/1577] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.405876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.410442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.411318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.411601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.411880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] PATH_INFO: `/v3/projects/b9c9dd4556f443508b4d63a0001b0111/users/21ea6a5a7cd347229a1755f68cc70f73/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.427998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c9dd4556f443508b4d63a0001b0111, user_id=21ea6a5a7cd347229a1755f68cc70f73, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.432600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:55.432600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:42:55.432600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:42:55.432600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.468616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb593a-14bd-4153-a2cc-422cc47da2d4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.473304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.473964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.474253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.474535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.475222 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.476895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:55.476895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:42:55.476895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:42:55.478091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.517166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 770/1578] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:55 2022] PUT /identity/v3/projects/b9c9dd4556f443508b4d63a0001b0111/users/21ea6a5a7cd347229a1755f68cc70f73/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:55.524342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.581865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de106c2-27e5-4df7-88f8-2730b6b6753b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.586167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 809/1579] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.586167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.586833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.587623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.587934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.588217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.588945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.590843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.593412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b9f66-4b31-4955-af73-2e88dfd84d73 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.598188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.598843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.599132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.599416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.600095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.602851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.701232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 771/1580] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.701232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.712036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 810/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.712036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.719808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e9116-5bbd-47d6-a374-8153c6742d25 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.750675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:55.753347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:55.766512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099601a-2be9-4b8c-a6c2-a71b1f19b7cd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.771390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.772099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.772819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.773112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.773809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.775852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.777543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.778440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.778726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.779005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] PATH_INFO: `/v3/projects/152c2da7d3c24db1a0bb5387331b758b/users/24bb323d2a8a41bb8660c9027e86dc62/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.793859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152c2da7d3c24db1a0bb5387331b758b, user_id=24bb323d2a8a41bb8660c9027e86dc62, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.798712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.909474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 811/1582] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:55 2022] PUT /identity/v3/projects/152c2da7d3c24db1a0bb5387331b758b/users/24bb323d2a8a41bb8660c9027e86dc62/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:55.910842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 772/1583] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:55.910969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.912516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:55.929585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807050-1ef5-4fbc-ae06-7ccc8e723ac1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.983045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96b82c-ee73-4b9b-82fd-c2b0d6c99446 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:55.988650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.989006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39e605-0af7-4bd1-bec4-505f826065b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:55.989385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.989674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.989783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39e605-0af7-4bd1-bec4-505f826065b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:55.989954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.990061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39e605-0af7-4bd1-bec4-505f826065b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:55.990241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39e605-0af7-4bd1-bec4-505f826065b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:55.990651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.990964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39e605-0af7-4bd1-bec4-505f826065b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:55.992576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39e605-0af7-4bd1-bec4-505f826065b8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:55.992834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.002522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 812/1584] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.003685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.003999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.004275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.030414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] MFA Rules not processed for user `21ea6a5a7cd347229a1755f68cc70f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:56.031553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:56.083460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 773/1585] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:55 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:56.083460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.092491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e7473-4adb-45b4-8f31-1dc607d685f0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.097198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.097847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.098134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.098412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.099358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.101468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.112732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 813/1586] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:56.112732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.113109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 774/1587] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.113109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.116458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1588-55b4-42bb-a8ed-ac0faa888057 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.121653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.122323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.122619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.122885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.123599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.125657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.128971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058d90-5a00-42e4-b54c-790c9f925eec admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.135407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.136079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.136457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.136896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.137641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.139450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.152631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 775/1588] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.153821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.163854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804908fe-3f19-43d5-90b7-4f07d0afc675 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.168625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc190e-70a3-4e42-b9a8-e886cc797c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.169297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc190e-70a3-4e42-b9a8-e886cc797c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.169585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc190e-70a3-4e42-b9a8-e886cc797c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.169868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc190e-70a3-4e42-b9a8-e886cc797c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.170597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc190e-70a3-4e42-b9a8-e886cc797c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.172324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc190e-70a3-4e42-b9a8-e886cc797c41 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.182728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 776/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.184083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.186935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.186935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.212964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] MFA Rules not processed for user `24bb323d2a8a41bb8660c9027e86dc62`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:56.214171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:56.224032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 814/1590] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:42:56 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:42:56.224032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.232454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5c5b2-76a9-4249-b81d-a07a70383317 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.237788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.238902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.239190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.239476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.240190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.242384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.256344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 815/1591] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.257817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.269966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298ae65-ae15-4c41-9e75-0e95945247d2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.275389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb060-777a-4429-acd0-937dde11dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.276044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb060-777a-4429-acd0-937dde11dd25 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.276335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb060-777a-4429-acd0-937dde11dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.276614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb060-777a-4429-acd0-937dde11dd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.277370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb060-777a-4429-acd0-937dde11dd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.278963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb060-777a-4429-acd0-937dde11dd25 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.285332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 777/1592] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:56.286166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.297190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 816/1593] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.297190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755718a-6b53-481a-8734-4caa8abbd2cb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.297190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755718a-6b53-481a-8734-4caa8abbd2cb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.297190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755718a-6b53-481a-8734-4caa8abbd2cb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.297609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80019b-f818-4a69-a23a-fd63de8873de None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.298440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 817/1594] 10.222.0.47 () {54 vars in 962 bytes} [Mon Jul 18 10:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:42:56.299215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755718a-6b53-481a-8734-4caa8abbd2cb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.302074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.307465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.307465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.307465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.307465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.307465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.313045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755718a-6b53-481a-8734-4caa8abbd2cb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.320078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 778/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.320078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.320309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.320309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.320309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.320309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.320598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.321866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.323521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:56.330671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc6587-a7d6-4032-89e0-eb85dc529c2f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.337103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.337974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.338258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.338536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.339259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.339376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 818/1596] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.342821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.342935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.345742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:56.349936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89ce5a-a72b-483f-94c0-351c1f57e604 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.354778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.355436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.355721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.355948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 779/1597] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.356081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.356800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.356967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.364185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.366805 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd642e2-fb11-4222-8c26-4edbf97180c5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.369716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 819/1598] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.370525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.371808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.372763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.373007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.373428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] PATH_INFO: `/v3/projects/f76bdb50a5c540e9aeb2099b993d334b/users/46d0d621460b4e5b877ef421d3503340/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.380137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5de00-bdfa-4956-b899-671f1f69071f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.385686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.386334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.386629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.386914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.387692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.389324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.389670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76bdb50a5c540e9aeb2099b993d334b, user_id=46d0d621460b4e5b877ef421d3503340, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.392443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.401003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 820/1599] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.404243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.453989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af950f-96d7-4284-9b46-974fc5e9ade3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.457984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbddf2e-2ad4-4efb-ba16-f75b57cdd59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.458622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbddf2e-2ad4-4efb-ba16-f75b57cdd59b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.458911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbddf2e-2ad4-4efb-ba16-f75b57cdd59b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.459194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbddf2e-2ad4-4efb-ba16-f75b57cdd59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.468138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbddf2e-2ad4-4efb-ba16-f75b57cdd59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.469604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbddf2e-2ad4-4efb-ba16-f75b57cdd59b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.478462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 821/1600] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:56.479633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.479992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.501796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.502291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 780/1601] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:56 2022] PUT /identity/v3/projects/f76bdb50a5c540e9aeb2099b993d334b/users/46d0d621460b4e5b877ef421d3503340/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:56.502291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.526845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:56.528037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:56.555777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2880-03ff-4259-acf3-cda5eca594f9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.560807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.561784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.562143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.562611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] PATH_INFO: `/v3/projects/0ff7695de0d44e55a0f1b7396c3fd0e7/users/74d7e32d47864c9a8294d8c4dd954618/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.580964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff7695de0d44e55a0f1b7396c3fd0e7, user_id=74d7e32d47864c9a8294d8c4dd954618, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.583654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.598949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 822/1602] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Jul 18 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:56.599782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.610980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d43ef-76c2-4501-b9d1-a09a8cc4bc9f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.615549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.617710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.617710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.617710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] PATH_INFO: `/v3/projects/b0e325a6638442b9a2a929a6525f7c2e/users/b86d6011a3f04ad0bf5b8be696c789f2/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.638785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e325a6638442b9a2a929a6525f7c2e, user_id=b86d6011a3f04ad0bf5b8be696c789f2, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.641465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.668538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 781/1603] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:56 2022] PUT /identity/v3/projects/0ff7695de0d44e55a0f1b7396c3fd0e7/users/74d7e32d47864c9a8294d8c4dd954618/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:56.676293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.739261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87219b1a-7a6c-48ae-b553-33f256d1fd56 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.744908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616d6b2-15b9-4ed0-af01-c4b3f84aa2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.745798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616d6b2-15b9-4ed0-af01-c4b3f84aa2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.746087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616d6b2-15b9-4ed0-af01-c4b3f84aa2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.746367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616d6b2-15b9-4ed0-af01-c4b3f84aa2b0 admin admin] PATH_INFO: `/v3/projects/f5f3aeaf69e147fc9c2e04723cea0fd2/users/8eb0ed48dd034be0ad63373e4a151cd9/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.760604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 823/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:56 2022] PUT /identity/v3/projects/b0e325a6638442b9a2a929a6525f7c2e/users/b86d6011a3f04ad0bf5b8be696c789f2/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:56.764306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.764539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616d6b2-15b9-4ed0-af01-c4b3f84aa2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f3aeaf69e147fc9c2e04723cea0fd2, user_id=8eb0ed48dd034be0ad63373e4a151cd9, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.765223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616d6b2-15b9-4ed0-af01-c4b3f84aa2b0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.794062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:56.822748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735584c7-7108-41cb-a0bb-21d0829e3bae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.827292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.827951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.828236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.828510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.829278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.831140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.840965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 824/1605] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:42:56.841746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:56.854399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba12a-c220-49f3-a2ae-f81d22d5cb79 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:56.858243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 782/1606] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:56 2022] PUT /identity/v3/projects/f5f3aeaf69e147fc9c2e04723cea0fd2/users/8eb0ed48dd034be0ad63373e4a151cd9/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:56.862117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.862117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.862117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.878321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:56.879526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:56.912248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd936dee-e2d6-4408-ad8e-5abc80941a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:56.913740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd936dee-e2d6-4408-ad8e-5abc80941a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:56.914026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd936dee-e2d6-4408-ad8e-5abc80941a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:56.914306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd936dee-e2d6-4408-ad8e-5abc80941a56 admin admin] PATH_INFO: `/v3/projects/ad323d7f7c484b5bb161d92791c9c7d7/users/df7a3101955b481f9a582168731ae76c/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:56.914929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:56.929536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd936dee-e2d6-4408-ad8e-5abc80941a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad323d7f7c484b5bb161d92791c9c7d7, user_id=df7a3101955b481f9a582168731ae76c, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:56.932224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd936dee-e2d6-4408-ad8e-5abc80941a56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:56.940483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 783/1607] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:56.941500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.013874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0aae5-f781-4eb2-ae7a-4656f1af109e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.019682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e68494-530e-44de-8001-3c1fef22d8e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.044870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e68494-530e-44de-8001-3c1fef22d8e5 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.044870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e68494-530e-44de-8001-3c1fef22d8e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.044870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e68494-530e-44de-8001-3c1fef22d8e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.084826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 825/1608] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:56 2022] PUT /identity/v3/projects/ad323d7f7c484b5bb161d92791c9c7d7/users/df7a3101955b481f9a582168731ae76c/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:57.084826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7592bb-43c1-476f-8fa9-b152765a7a60 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.084826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7592bb-43c1-476f-8fa9-b152765a7a60 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.084826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7592bb-43c1-476f-8fa9-b152765a7a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.085321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e68494-530e-44de-8001-3c1fef22d8e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.085321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e68494-530e-44de-8001-3c1fef22d8e5 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.092041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 784/1609] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.093351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.093644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.093925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.105391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8c7592bb-43c1-476f-8fa9-b152765a7a60 None None] MFA Rules not processed for user `74d7e32d47864c9a8294d8c4dd954618`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.106781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8c7592bb-43c1-476f-8fa9-b152765a7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.111983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.113384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.174901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 785/1610] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.175573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.179503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 826/1611] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.186793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c163a5-a858-4855-899d-faac6f65f9c6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.192574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae35e-9a11-4cee-be5e-d7fbf6b4082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.193509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae35e-9a11-4cee-be5e-d7fbf6b4082e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.193787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae35e-9a11-4cee-be5e-d7fbf6b4082e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.194101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae35e-9a11-4cee-be5e-d7fbf6b4082e admin admin] PATH_INFO: `/v3/projects/9e4aa903be8d4afc91f368051333848b/users/17e0c6def385441bbdde18c63b7d99e3/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.199211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.199510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.200273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.218423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae35e-9a11-4cee-be5e-d7fbf6b4082e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4aa903be8d4afc91f368051333848b, user_id=17e0c6def385441bbdde18c63b7d99e3, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.222890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae35e-9a11-4cee-be5e-d7fbf6b4082e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.234045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.234159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:57.234263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.238858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 827/1612] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.240041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.285377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:57.305814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5d1ce-16cf-45e7-8305-e324bfd5909b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.309923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.310569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.310858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.311146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.318413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 786/1613] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:42:57 2022] PUT /identity/v3/projects/9e4aa903be8d4afc91f368051333848b/users/17e0c6def385441bbdde18c63b7d99e3/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:57.318413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ed29-1c5a-482b-9fc0-20d5c089346a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.318413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ed29-1c5a-482b-9fc0-20d5c089346a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.318413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ed29-1c5a-482b-9fc0-20d5c089346a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.323447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.325368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.336147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 828/1614] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.336727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.338011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-57b4ed29-1c5a-482b-9fc0-20d5c089346a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.341109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.341798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-57b4ed29-1c5a-482b-9fc0-20d5c089346a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.359728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aa6f3-e2f1-4891-bc73-dad7b42a9234 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.408732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70c525-31f4-4608-872d-55e0a129c00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.409389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70c525-31f4-4608-872d-55e0a129c00e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.409678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70c525-31f4-4608-872d-55e0a129c00e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.409961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70c525-31f4-4608-872d-55e0a129c00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.410613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 787/1615] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.412001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba431-cad5-49f6-aa88-249348ee9646 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.412301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba431-cad5-49f6-aa88-249348ee9646 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.412571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba431-cad5-49f6-aa88-249348ee9646 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.414782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 788/1616] 10.222.0.47 () {54 vars in 962 bytes} [Mon Jul 18 10:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:42:57.417564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba431-cad5-49f6-aa88-249348ee9646 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.423906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70c525-31f4-4608-872d-55e0a129c00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.428820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70c525-31f4-4608-872d-55e0a129c00e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.429519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba431-cad5-49f6-aa88-249348ee9646 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.433144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.433773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.434067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.434344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.435757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 829/1617] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.437073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7d426-4f9e-4264-b65a-ae231a32b9de None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.437315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7d426-4f9e-4264-b65a-ae231a32b9de None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.437591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7d426-4f9e-4264-b65a-ae231a32b9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.444347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.445890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.456471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e1f7d426-4f9e-4264-b65a-ae231a32b9de None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.463223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1f7d426-4f9e-4264-b65a-ae231a32b9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.470243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 830/1618] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.474562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ae5b-8ab2-4ba5-b707-295ef72c2c84 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.475017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ae5b-8ab2-4ba5-b707-295ef72c2c84 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.475327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ae5b-8ab2-4ba5-b707-295ef72c2c84 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.477567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 831/1619] 10.222.0.47 () {54 vars in 912 bytes} [Mon Jul 18 10:42:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:42:57.478116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 789/1620] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.478694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.478980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.479262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.479814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.501026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92059fd0-9729-4714-9315-3ae99724b668 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.504002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.505348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.505612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441ba2d-8e9f-44c2-9100-483dcacdb5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.506275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ba2d-8e9f-44c2-9100-483dcacdb5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.506751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ba2d-8e9f-44c2-9100-483dcacdb5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.507378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ba2d-8e9f-44c2-9100-483dcacdb5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.518761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441ba2d-8e9f-44c2-9100-483dcacdb5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.520316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441ba2d-8e9f-44c2-9100-483dcacdb5a6 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.532812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 790/1621] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.534566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.535012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.535346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.560406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:57.561848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:57.647094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 832/1622] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.649448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.662240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a59633-a007-4f7b-b015-ee2272be8989 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.673580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.673580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.673580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.673580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.674016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 791/1623] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:57.674016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.682239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d647a-1aa9-42a1-8eef-88e66e7658ce None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.685832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.687401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.698750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f8041-fa82-4d3c-9fa2-08ad983359fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.699506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f8041-fa82-4d3c-9fa2-08ad983359fa service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.699902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f8041-fa82-4d3c-9fa2-08ad983359fa service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.700012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 833/1624] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.700359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f8041-fa82-4d3c-9fa2-08ad983359fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.702093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:57.778468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af59a46-c457-45bd-8a79-1b4fea22db8d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:57.783167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c90375-3c39-4128-8e32-711d50faf2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:57.783839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90375-3c39-4128-8e32-711d50faf2fd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:57.784132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90375-3c39-4128-8e32-711d50faf2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:57.784411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90375-3c39-4128-8e32-711d50faf2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:57.796624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c90375-3c39-4128-8e32-711d50faf2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.798531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c90375-3c39-4128-8e32-711d50faf2fd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.820479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f8041-fa82-4d3c-9fa2-08ad983359fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:57.823055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 834/1625] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:57.825058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f8041-fa82-4d3c-9fa2-08ad983359fa service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:57.832906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:57.858627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 792/1626] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:58.180898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914fe3-690b-4314-8d28-975323eca8db None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:58.184200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914fe3-690b-4314-8d28-975323eca8db None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:58.184200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914fe3-690b-4314-8d28-975323eca8db None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:58.184200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 835/1627] 10.222.0.47 () {54 vars in 912 bytes} [Mon Jul 18 10:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:42:58.189097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:58.189585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:58.189987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:58.211730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:58.213885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:58.224830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 793/1628] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:58.241816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a914fe3-690b-4314-8d28-975323eca8db None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:58.274315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a914fe3-690b-4314-8d28-975323eca8db None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:58.278492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ecb94-0cf5-4e19-92c4-f650d8da4ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:58.279429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ecb94-0cf5-4e19-92c4-f650d8da4ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:58.280063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ecb94-0cf5-4e19-92c4-f650d8da4ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:58.280624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ecb94-0cf5-4e19-92c4-f650d8da4ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:58.295426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ecb94-0cf5-4e19-92c4-f650d8da4ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:58.297454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ecb94-0cf5-4e19-92c4-f650d8da4ef8 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:58.324882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 836/1629] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:42:58.384091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:42:58.515400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:58.520618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d8d41-a36a-48bf-82a8-d4f4fd2e5f58 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:58.527926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:58.528932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:58.529265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:58.529734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] PATH_INFO: `/v3/users/74d7e32d47864c9a8294d8c4dd954618` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:58.531006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d7e32d47864c9a8294d8c4dd954618)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:58.532211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:42:58.532211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jul 18 10:42:58.532211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: Jul 18 10:42:58.533812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:59.142668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d7e32d47864c9a8294d8c4dd954618'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:59.142668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d7e32d47864c9a8294d8c4dd954618'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:59.142668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d7e32d47864c9a8294d8c4dd954618'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:59.142668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d7e32d47864c9a8294d8c4dd954618'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:59.142668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d7e32d47864c9a8294d8c4dd954618'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:42:59.168848 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 794/1630] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:42:58 2022] DELETE /identity/v3/users/74d7e32d47864c9a8294d8c4dd954618 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:42:59.170000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:59.170000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:59.170000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:59.170000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:42:59.170000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:59.201337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 837/1631] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:42:59.213137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:42:59.214656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:42:59.281703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564c9c0-080a-42b0-9235-15a01e0a61dc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:42:59.285474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898388b-0a5c-4c4f-aa43-c43b7cfcb579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:42:59.286126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898388b-0a5c-4c4f-aa43-c43b7cfcb579 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:42:59.286418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898388b-0a5c-4c4f-aa43-c43b7cfcb579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:42:59.286702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898388b-0a5c-4c4f-aa43-c43b7cfcb579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:42:59.296081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898388b-0a5c-4c4f-aa43-c43b7cfcb579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:42:59.297546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898388b-0a5c-4c4f-aa43-c43b7cfcb579 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:42:59.307178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 795/1632] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:00.120833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:00.123037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:00.153528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a6878-d2ba-4b27-9550-5f1d6613b348 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:00.158927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:00.160100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:00.160483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:00.161572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] PATH_INFO: `/v3/projects/0ff7695de0d44e55a0f1b7396c3fd0e7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:00.186650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff7695de0d44e55a0f1b7396c3fd0e7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:00.189159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:43:00.189159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 18 10:43:00.189159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: Jul 18 10:43:00.192367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:00.415052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff7695de0d44e55a0f1b7396c3fd0e7', 'request_id': 'req-b77c3cb4-fbb3-432a-9f13-8424175d43fc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:00.431388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff7695de0d44e55a0f1b7396c3fd0e7', 'request_id': 'req-b77c3cb4-fbb3-432a-9f13-8424175d43fc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:00.574512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff7695de0d44e55a0f1b7396c3fd0e7', 'request_id': 'req-b77c3cb4-fbb3-432a-9f13-8424175d43fc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:00.597352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] The token cache is being invalidate because project 0ff7695de0d44e55a0f1b7396c3fd0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:00.597736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:00.599327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b77c3cb4-fbb3-432a-9f13-8424175d43fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:00.608990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 838/1633] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:00 2022] DELETE /identity/v3/projects/0ff7695de0d44e55a0f1b7396c3fd0e7 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:00.616549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:00.619748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:00.621111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:00.684282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:00.685790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:00.766806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 796/1634] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:00.775430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:00.775969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:00.776389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:00.816606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:00.819617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:00.829414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 839/1635] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:00.834077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:00.850476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f994eb-2f04-4007-ba75-b98a11ee89dc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:00.857755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:00.859306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:00.861775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:00.862251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:00.863298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:00.864813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:00.866890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:00.869470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:00.883611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 797/1636] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:00.887696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:00.908075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97919f31-0760-48e1-a32c-90c3cb2a1316 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:00.914315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:00.915193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:00.915591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:00.915968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:00.916890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:00.920571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:01.049502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 840/1637] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:00 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:01.054950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:01.065917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f40ae-5c71-4387-982c-df0e51cd24fb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:01.152890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:01.153832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.154489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.154875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.155780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:01.158272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:01.282933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 798/1638] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:01 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:43:01.289888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:01.326492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd12447-25f0-40e0-a058-3a8679e293a0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:01.345782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:01.346918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.347575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.348020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.349012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:01.352002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:01.374469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 841/1639] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:01.378926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:01.391915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158002-0c4c-45a6-8042-c41dc4e3a956 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:01.397696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3955272-745e-4834-aadd-5947f72f7502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:01.399684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955272-745e-4834-aadd-5947f72f7502 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.400277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955272-745e-4834-aadd-5947f72f7502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.401098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955272-745e-4834-aadd-5947f72f7502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.403983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3955272-745e-4834-aadd-5947f72f7502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:01.404570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3955272-745e-4834-aadd-5947f72f7502 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:01.420435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 799/1640] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:01.421366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:01.437573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74eeb1-e38f-4f74-86d6-814d070e2d6e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:01.449435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:01.450219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.450513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.450845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] PATH_INFO: `/v3/projects/befdf60d5e0f4578916838247b1d13ce/users/7b28e20a0e7948df9ce2d9bb17ca254a/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.471698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=befdf60d5e0f4578916838247b1d13ce, user_id=7b28e20a0e7948df9ce2d9bb17ca254a, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:01.475088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:01.578656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:01.586738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 842/1641] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:01 2022] PUT /identity/v3/projects/befdf60d5e0f4578916838247b1d13ce/users/7b28e20a0e7948df9ce2d9bb17ca254a/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:01.598613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.599125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.599546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.648722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] MFA Rules not processed for user `7b28e20a0e7948df9ce2d9bb17ca254a`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:01.652720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:01.669533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:01.672656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:01.839841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 800/1642] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:01.841742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9337f-85dd-4ea4-8b10-ecf5710e2552 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:01.843992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:01.854015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:01.855391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.855743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.857213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] PATH_INFO: `/v3/users/21ea6a5a7cd347229a1755f68cc70f73` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.858848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ea6a5a7cd347229a1755f68cc70f73)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:01.874113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:01.900013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660113b-f1da-4391-83c4-b90d0a5c46b1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:01.920484 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:01.921325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:01.923183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:01.923183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:01.931647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:01.937674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.106776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 801/1643] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:01 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:43:02.111549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.151514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ea6a5a7cd347229a1755f68cc70f73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:02.167235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ea6a5a7cd347229a1755f68cc70f73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:02.204555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff8ccb-d735-4302-902e-f9d48159e3a5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.210288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:02.210945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.211300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.212017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.212463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:02.214502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.308514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ea6a5a7cd347229a1755f68cc70f73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:02.310464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ea6a5a7cd347229a1755f68cc70f73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:02.313467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e8a05e2a-bba5-427c-99ac-317cb0341fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ea6a5a7cd347229a1755f68cc70f73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:02.358300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 802/1644] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:02 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:02.359380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.367836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 843/1645] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:01 2022] DELETE /identity/v3/users/21ea6a5a7cd347229a1755f68cc70f73 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:02.368287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:02.372278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.372918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.373348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.393152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd6b38-32a1-4d64-bb4c-97744b89ea9e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.426159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:02.429908 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:02.522820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:02.524943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.525961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.527015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.529711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:02.535168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.550811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 844/1646] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:02.554982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 803/1647] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:02.559900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.577556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7e84-08a9-4b08-a2f9-f9dc0c5be1a0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.592505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:02.593772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.593772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.594064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] PATH_INFO: `/v3/projects/7ef73b6dc97e48e096b9ce0ec84c4c89/users/2c25ddb2d79447d0bf0da1735eb62357/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.619653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef73b6dc97e48e096b9ce0ec84c4c89, user_id=2c25ddb2d79447d0bf0da1735eb62357, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:02.624757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.745856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.747677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:02.752750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 845/1648] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:02 2022] PUT /identity/v3/projects/7ef73b6dc97e48e096b9ce0ec84c4c89/users/2c25ddb2d79447d0bf0da1735eb62357/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:02.753813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.860900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbae4e-dd44-4ec7-8800-47415ea18299 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.891849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9c485-05d4-4e6c-972f-74aafd0f002c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.901828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:02.902578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.903073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.903365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.904171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:02.905892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.920803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 846/1649] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:02.926093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.950568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8220d-d65e-47af-9080-f829e845f1a2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.962665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f87c00-e415-4067-94fb-9987765dcc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:02.963812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f87c00-e415-4067-94fb-9987765dcc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.964107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f87c00-e415-4067-94fb-9987765dcc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.964601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f87c00-e415-4067-94fb-9987765dcc7d admin admin] PATH_INFO: `/v3/projects/7ef73b6dc97e48e096b9ce0ec84c4c89/users/2c25ddb2d79447d0bf0da1735eb62357/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.967147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f87c00-e415-4067-94fb-9987765dcc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef73b6dc97e48e096b9ce0ec84c4c89, user_id=2c25ddb2d79447d0bf0da1735eb62357, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:02.970852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f87c00-e415-4067-94fb-9987765dcc7d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.988472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 847/1650] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:02 2022] PUT /identity/v3/projects/7ef73b6dc97e48e096b9ce0ec84c4c89/users/2c25ddb2d79447d0bf0da1735eb62357/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:03.001985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:03.002688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:03.009542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:03.009542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:03.009740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:03.009740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:03.009740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:03.037321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:03.039029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:03.066138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 804/1651] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:03.069960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] MFA Rules not processed for user `2c25ddb2d79447d0bf0da1735eb62357`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:03.071851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:03.222382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 848/1652] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:03.233218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:03.234637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:03.405307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:03.408650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b82f9-518a-43b8-9c30-1e1fbe044fdf service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:03.486609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe45635-5fd4-42ce-a8fe-f746bf48dba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:03.487447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45635-5fd4-42ce-a8fe-f746bf48dba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:03.487746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45635-5fd4-42ce-a8fe-f746bf48dba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:03.488162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45635-5fd4-42ce-a8fe-f746bf48dba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:03.524714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45635-5fd4-42ce-a8fe-f746bf48dba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:03.530570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45635-5fd4-42ce-a8fe-f746bf48dba4 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:03.564305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 805/1653] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:04.469064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:04.696510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:04.698599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:04.994420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f580-9e00-4bdb-b654-9958d15490b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:05.005433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:05.015046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:05.015046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:05.015046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:05.015508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] PATH_INFO: `/v3/projects/b9c9dd4556f443508b4d63a0001b0111` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:05.017742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c9dd4556f443508b4d63a0001b0111)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:05.019465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:05.264822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c9dd4556f443508b4d63a0001b0111', 'request_id': 'req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:05.302173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c9dd4556f443508b4d63a0001b0111', 'request_id': 'req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:05.497736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c9dd4556f443508b4d63a0001b0111', 'request_id': 'req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:05.524189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] The token cache is being invalidate because project b9c9dd4556f443508b4d63a0001b0111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:05.524382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:05.532841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe347a0a-c3c6-43a4-a4dd-9c32b0765fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:05.543159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 849/1654] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:04 2022] DELETE /identity/v3/projects/b9c9dd4556f443508b4d63a0001b0111 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:05.544382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:05.545022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:05.545503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:05.604090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:05.605677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:05.887301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:05.887818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:05.888304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:05.897521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 806/1655] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:05.925831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:05.930801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:05.952538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:05.953105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 850/1656] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:05.965469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85403d-9ab7-4471-913e-edca6aee5a3a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:05.970979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:05.971705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:05.972008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:05.972291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:05.973177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:05.973493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:05.976124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:05.996067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:06.017542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 807/1657] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:06.020010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a6afe-aa39-4aed-9915-ee164b5e84dd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:06.033408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:06.034104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:06.036501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:06.037187 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:06.038185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:06.040130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:06.049572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:06.175502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 851/1658] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:05 2022] POST /identity/v3/projects => generated 405 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:06.181005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:06.202765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5934903-1fff-4b36-8909-83772b0d7002 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:06.526732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:06.527520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:06.527812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:06.528098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:06.528876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:06.539431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:06.678985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:06.698477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 808/1659] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:06 2022] POST /identity/v3/users => generated 303 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:06.701476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:06.729025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a0e53-6425-4779-ac3d-99014e159015 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:06.746961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:06.748425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:06.748993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:06.749832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:06.752716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:06.761143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:06.802178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:06.803152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 852/1660] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:06.824282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d09d7-6918-41e2-9fea-f65b0aed2754 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:06.838013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11609-61e4-488a-a9a6-e0a5061f48e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:06.838771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11609-61e4-488a-a9a6-e0a5061f48e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:06.839238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11609-61e4-488a-a9a6-e0a5061f48e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:06.839526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11609-61e4-488a-a9a6-e0a5061f48e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:06.840271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11609-61e4-488a-a9a6-e0a5061f48e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:06.845052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11609-61e4-488a-a9a6-e0a5061f48e3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:06.886024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:06.899367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 809/1661] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:06.904423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a875df-1624-4990-96c6-7bdc249eb0ad admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:06.947554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:06.950894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:06.954869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:06.954869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] PATH_INFO: `/v3/projects/9515615d70d84b049ceceeaba8a63ff8/users/2aa054051aa143d7a73cb95367e3b3a1/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:07.017339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9515615d70d84b049ceceeaba8a63ff8, user_id=2aa054051aa143d7a73cb95367e3b3a1, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:07.036249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:07.182284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 853/1662] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:06 2022] PUT /identity/v3/projects/9515615d70d84b049ceceeaba8a63ff8/users/2aa054051aa143d7a73cb95367e3b3a1/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:07.193370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:07.194808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:07.195278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:07.239840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] MFA Rules not processed for user `2aa054051aa143d7a73cb95367e3b3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:07.241543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:07.336874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:07.340190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 810/1663] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:07.450856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612b72-33c4-4fbe-afe2-23f0fe4046d6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:07.461262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:07.462025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:07.462646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:07.463039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:07.464398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:07.470761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:07.621259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 854/1664] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:07 2022] POST /identity/v3/projects => generated 403 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:43:07.626360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:07.647910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af03e3c-66f0-4dad-9b60-0a7926b29b69 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:07.811098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:07.813357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:07.813357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:07.813357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:07.813357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:07.827963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:07.920955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 811/1665] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:07 2022] POST /identity/v3/users => generated 308 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:07.924625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:07.948111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24d2c-0bee-47e6-ba52-cac63d198fc3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:07.972640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:07.977393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:07.977689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:07.980724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:07.982719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:07.995104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:08.028132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 855/1666] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:08.032444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:08.074760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3bd25-2c8b-404d-81ed-c8ccf36156aa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:08.090427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:08.091474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:08.098258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:08.098258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] PATH_INFO: `/v3/projects/012d42cfc6394ee2861cd66ad8be5a2c/users/e3c03360d9bb40fc9bef177925748a18/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:08.139624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] RBAC: Authorizing `identity:create_grant(project_id=012d42cfc6394ee2861cd66ad8be5a2c, user_id=e3c03360d9bb40fc9bef177925748a18, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:08.142558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:08.263518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 812/1667] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:08 2022] PUT /identity/v3/projects/012d42cfc6394ee2861cd66ad8be5a2c/users/e3c03360d9bb40fc9bef177925748a18/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:08.265064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:08.273742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:08.282840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277f82-08d5-4d92-9798-07d67f928603 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:08.418899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b46c9-e87d-45f0-b8f3-7e0211e06e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:08.420082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b46c9-e87d-45f0-b8f3-7e0211e06e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:08.420544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b46c9-e87d-45f0-b8f3-7e0211e06e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:08.421667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b46c9-e87d-45f0-b8f3-7e0211e06e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:08.423931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b46c9-e87d-45f0-b8f3-7e0211e06e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:08.426496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b46c9-e87d-45f0-b8f3-7e0211e06e56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:08.489582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 856/1668] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:08.494887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:08.528670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e3d61-850f-4957-86f7-b0bcc6d3512e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:08.552395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:08.556585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:08.557346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:08.566332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] PATH_INFO: `/v3/projects/012d42cfc6394ee2861cd66ad8be5a2c/users/e3c03360d9bb40fc9bef177925748a18/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:08.568877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] RBAC: Authorizing `identity:create_grant(project_id=012d42cfc6394ee2861cd66ad8be5a2c, user_id=e3c03360d9bb40fc9bef177925748a18, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:08.571896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:08.596436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 813/1669] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:08 2022] PUT /identity/v3/projects/012d42cfc6394ee2861cd66ad8be5a2c/users/e3c03360d9bb40fc9bef177925748a18/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:08.616642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c60fd7-48b9-4bc1-bea6-7f23d1357cff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:08.617929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c60fd7-48b9-4bc1-bea6-7f23d1357cff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:08.618392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c60fd7-48b9-4bc1-bea6-7f23d1357cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:08.736922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-37c60fd7-48b9-4bc1-bea6-7f23d1357cff None None] MFA Rules not processed for user `e3c03360d9bb40fc9bef177925748a18`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:08.739330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37c60fd7-48b9-4bc1-bea6-7f23d1357cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:09.010818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 857/1670] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:09.019190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:09.021152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:09.109902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:09.170126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4b59c-3572-4d35-8af4-855eb650767f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:09.174441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:09.175123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:09.175408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:09.175878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:09.218764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:09.222359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:09.277920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 814/1671] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:09.493436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:09.493862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:09.494267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:09.510767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:09.512433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:09.592130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] MFA Rules not processed for user `b86d6011a3f04ad0bf5b8be696c789f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:09.601421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:09.645665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79ee77-220a-4c6e-b5e9-d258d64d4ef6 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:09.658302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:09.659607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:09.659607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:09.659946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] PATH_INFO: `/v3/users/7b28e20a0e7948df9ce2d9bb17ca254a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:09.662659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b28e20a0e7948df9ce2d9bb17ca254a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:09.665674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:09.833461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 858/1672] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:09.834510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:09.855692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c31401-29a6-49af-8832-76defc47a952 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:09.860743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:09.861676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:09.861963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:09.862247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] PATH_INFO: `/v3/users/2aa054051aa143d7a73cb95367e3b3a1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:09.870326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa054051aa143d7a73cb95367e3b3a1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:09.873676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:10.014942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b28e20a0e7948df9ce2d9bb17ca254a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.051779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b28e20a0e7948df9ce2d9bb17ca254a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.073352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b28e20a0e7948df9ce2d9bb17ca254a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.175259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa054051aa143d7a73cb95367e3b3a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.176436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b28e20a0e7948df9ce2d9bb17ca254a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.177108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0c63614c-6940-401b-92e6-9113785d1541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b28e20a0e7948df9ce2d9bb17ca254a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.185713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 815/1673] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:09 2022] DELETE /identity/v3/users/7b28e20a0e7948df9ce2d9bb17ca254a => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:10.187116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c48dc-4b5d-4504-8af9-63e37e2ecb7e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:10.189518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c48dc-4b5d-4504-8af9-63e37e2ecb7e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:10.189518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c48dc-4b5d-4504-8af9-63e37e2ecb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:10.189687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa054051aa143d7a73cb95367e3b3a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.237365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cd7c48dc-4b5d-4504-8af9-63e37e2ecb7e None None] MFA Rules not processed for user `b86d6011a3f04ad0bf5b8be696c789f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:10.238890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cd7c48dc-4b5d-4504-8af9-63e37e2ecb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:10.285548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa054051aa143d7a73cb95367e3b3a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.286748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa054051aa143d7a73cb95367e3b3a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.289935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa054051aa143d7a73cb95367e3b3a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:10.342241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 816/1674] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:10.344321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c9edd-18a2-4445-806b-421ccc352a63 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:10.344804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c9edd-18a2-4445-806b-421ccc352a63 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:10.345245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c9edd-18a2-4445-806b-421ccc352a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:10.351200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 859/1675] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:09 2022] DELETE /identity/v3/users/2aa054051aa143d7a73cb95367e3b3a1 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:10.355211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:10.359520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:10.377646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c02c9edd-18a2-4445-806b-421ccc352a63 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:10.389353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c02c9edd-18a2-4445-806b-421ccc352a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:10.565283 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 817/1676] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:10.567148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:10.567834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:10.568320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:10.622589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e5349-1cb4-43d4-a322-5bf448fa131a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:10.641833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:10.643395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:10.647836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:10.648594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:10.648913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:10.649245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:10.657417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:10.667475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 818/1677] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:10.668660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:10.682327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:10.802640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016843d-9cf2-488a-bf0a-881550cfcf24 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:10.803917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:10.807259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:10.815038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:10.816717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:10.817206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:10.817676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:10.845809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 860/1678] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:10.851189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:10.854324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:10.858420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:10.881933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 819/1679] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:10.883163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e377f-9051-455b-85a9-03f6465049e4 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:10.895107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415341c6-de3a-4f9b-a8e4-92eb8604888f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:10.895953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415341c6-de3a-4f9b-a8e4-92eb8604888f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:10.897350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415341c6-de3a-4f9b-a8e4-92eb8604888f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:10.897829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415341c6-de3a-4f9b-a8e4-92eb8604888f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:10.923846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415341c6-de3a-4f9b-a8e4-92eb8604888f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:10.934697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415341c6-de3a-4f9b-a8e4-92eb8604888f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:10.997023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 861/1680] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:11.203905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:11.295870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:11.468077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:11.473534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:11.518803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e652bd-1bd0-42ed-bb48-85b59e97236f service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:11.533747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:11.534836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:11.538365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:11.538814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] PATH_INFO: `/v3/users/24bb323d2a8a41bb8660c9027e86dc62` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:11.540333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bb323d2a8a41bb8660c9027e86dc62)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:11.542444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:11.750205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6bbfd-9423-4316-b700-5fb5c1054775 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:11.750503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6bbfd-9423-4316-b700-5fb5c1054775 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:11.750780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6bbfd-9423-4316-b700-5fb5c1054775 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:11.752181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 862/1681] 10.222.0.47 () {54 vars in 912 bytes} [Mon Jul 18 10:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:43:11.763352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840525f-1870-413d-87b8-1e8fc00d08a2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:11.764071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840525f-1870-413d-87b8-1e8fc00d08a2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:11.765261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840525f-1870-413d-87b8-1e8fc00d08a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:11.824134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bb323d2a8a41bb8660c9027e86dc62'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:11.870607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bb323d2a8a41bb8660c9027e86dc62'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:11.892323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bb323d2a8a41bb8660c9027e86dc62'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:11.914634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8840525f-1870-413d-87b8-1e8fc00d08a2 None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:11.916051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8840525f-1870-413d-87b8-1e8fc00d08a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:11.934874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 863/1682] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:43:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:11.999248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bb323d2a8a41bb8660c9027e86dc62'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:12.000223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bb323d2a8a41bb8660c9027e86dc62'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:12.008451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 820/1683] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:11 2022] DELETE /identity/v3/users/24bb323d2a8a41bb8660c9027e86dc62 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:12.015359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:12.015686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:12.015979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:12.048373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:12.049667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:12.082401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:12.132372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:12.137018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:12.293615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 864/1684] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:12.348567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e122cd1-66d0-46e7-b899-6ba0dec40d6d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:12.360510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:12.364015 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:12.364343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:12.364622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:12.373415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:12.374396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:12.385575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:12.402851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a35c7a-a87b-41cf-9b11-1b258c130405 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:12.418401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:12.419080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:12.419682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:12.426389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:12.427263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:12.430939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:12.470062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:12.472160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:12.476343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 821/1685] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:12.505669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 865/1686] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:13.248821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:13.252906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:13.276620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c46b4-4343-431f-84cf-4cdedde8b10c service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:13.290761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:13.291856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:13.292290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:13.297930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] PATH_INFO: `/v3/projects/befdf60d5e0f4578916838247b1d13ce` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:13.299561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=befdf60d5e0f4578916838247b1d13ce)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:13.301883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:13.482462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befdf60d5e0f4578916838247b1d13ce', 'request_id': 'req-37a2dca9-b29b-411f-95fe-d84377165d68'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:13.526380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:13.553989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befdf60d5e0f4578916838247b1d13ce', 'request_id': 'req-37a2dca9-b29b-411f-95fe-d84377165d68'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:13.560836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befdf60d5e0f4578916838247b1d13ce', 'request_id': 'req-37a2dca9-b29b-411f-95fe-d84377165d68'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:13.588533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] The token cache is being invalidate because project befdf60d5e0f4578916838247b1d13ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:13.588878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:13.593178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:13.599169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 822/1687] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:13 2022] DELETE /identity/v3/projects/befdf60d5e0f4578916838247b1d13ce => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:13.602001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:13.618158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d71a-e2a5-4be9-a669-fa1019cbbe5b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:13.674195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:13.675748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:13.792918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:13.794233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:13.794670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:13.795093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] PATH_INFO: `/v3/users/2c25ddb2d79447d0bf0da1735eb62357` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:13.799953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c25ddb2d79447d0bf0da1735eb62357)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:13.802578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:13.824380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2dca9-b29b-411f-95fe-d84377165d68 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:13.844250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:13.845288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:13.845649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:13.846135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] PATH_INFO: `/v3/projects/9515615d70d84b049ceceeaba8a63ff8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:13.851316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9515615d70d84b049ceceeaba8a63ff8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:13.855251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:14.061016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9515615d70d84b049ceceeaba8a63ff8', 'request_id': 'req-57f05f0b-ee5c-4f9d-af06-1ca181841f49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.096939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c25ddb2d79447d0bf0da1735eb62357'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.114460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c25ddb2d79447d0bf0da1735eb62357'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.192628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9515615d70d84b049ceceeaba8a63ff8', 'request_id': 'req-57f05f0b-ee5c-4f9d-af06-1ca181841f49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.204494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9515615d70d84b049ceceeaba8a63ff8', 'request_id': 'req-57f05f0b-ee5c-4f9d-af06-1ca181841f49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.222544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] The token cache is being invalidate because project 9515615d70d84b049ceceeaba8a63ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:14.223082 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.224273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-57f05f0b-ee5c-4f9d-af06-1ca181841f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.230511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 823/1688] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:13 2022] DELETE /identity/v3/projects/9515615d70d84b049ceceeaba8a63ff8 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:14.232532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11b11d-6667-451b-aebe-a0c7a819871c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:14.232988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11b11d-6667-451b-aebe-a0c7a819871c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:14.235741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11b11d-6667-451b-aebe-a0c7a819871c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:14.241610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 824/1689] 10.222.0.47 () {54 vars in 961 bytes} [Mon Jul 18 10:43:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:43:14.242999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11b11d-6667-451b-aebe-a0c7a819871c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:14.266036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4f11b11d-6667-451b-aebe-a0c7a819871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:14.335121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c25ddb2d79447d0bf0da1735eb62357'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.335938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c25ddb2d79447d0bf0da1735eb62357'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.336559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3ca4651-1a07-44be-8eaf-d520e30d4989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c25ddb2d79447d0bf0da1735eb62357'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.361804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:14.376284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 866/1690] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:13 2022] DELETE /identity/v3/users/2c25ddb2d79447d0bf0da1735eb62357 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:14.377992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:14.381998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:14.382447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:14.421522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:14.422818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:14.436969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11b11d-6667-451b-aebe-a0c7a819871c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:14.557666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 867/1691] 10.222.0.47 () {58 vars in 1058 bytes} [Mon Jul 18 10:43:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:14.558695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:14.560088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:14.631883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:14.633147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:14.633539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:14.633920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] PATH_INFO: `/v3/users/e3c03360d9bb40fc9bef177925748a18` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:14.638653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c03360d9bb40fc9bef177925748a18)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:14.640795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:14.718100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e062-92cd-47cb-8f9e-2c294ff36e6b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:14.728574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:14.729731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:14.730196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:14.730739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] PATH_INFO: `/v3/projects/152c2da7d3c24db1a0bb5387331b758b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:14.732897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] RBAC: Authorizing `identity:delete_project(project_id=152c2da7d3c24db1a0bb5387331b758b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:14.738054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:14.971623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152c2da7d3c24db1a0bb5387331b758b', 'request_id': 'req-d7424bfd-3213-48d3-96fc-bedd985d513b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.971950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c03360d9bb40fc9bef177925748a18'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:14.998785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152c2da7d3c24db1a0bb5387331b758b', 'request_id': 'req-d7424bfd-3213-48d3-96fc-bedd985d513b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.017410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c03360d9bb40fc9bef177925748a18'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.041658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c03360d9bb40fc9bef177925748a18'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.076893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152c2da7d3c24db1a0bb5387331b758b', 'request_id': 'req-d7424bfd-3213-48d3-96fc-bedd985d513b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.097490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] The token cache is being invalidate because project 152c2da7d3c24db1a0bb5387331b758b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:15.097794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.102506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7424bfd-3213-48d3-96fc-bedd985d513b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.107534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 868/1692] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:14 2022] DELETE /identity/v3/projects/152c2da7d3c24db1a0bb5387331b758b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:15.109250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:15.109677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:15.110086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:15.154205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] MFA Rules not processed for user `b86d6011a3f04ad0bf5b8be696c789f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:15.155824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:15.234368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c03360d9bb40fc9bef177925748a18'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.235551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5bb27c4-8587-43d6-a4c3-37a0644b6b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c03360d9bb40fc9bef177925748a18'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:15.246811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 869/1693] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:15.248015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:15.249525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:15.276795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 825/1694] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:14 2022] DELETE /identity/v3/users/e3c03360d9bb40fc9bef177925748a18 => generated 0 bytes in 1034 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:15.290423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8751a-c593-48a1-aac4-d2125c3be26f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:15.293641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8751a-c593-48a1-aac4-d2125c3be26f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:15.294280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8751a-c593-48a1-aac4-d2125c3be26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:15.372896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eee8751a-c593-48a1-aac4-d2125c3be26f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:15.380411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14d89-2722-4cd3-921d-2b5532f297ff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:15.382438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eee8751a-c593-48a1-aac4-d2125c3be26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:15.520789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:15.522671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:15.523412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:15.547240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:15.569899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:15.571067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 826/1695] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:15.583384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:15.584370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:15.585089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:15.628038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:15.631365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:15.643454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:15.649417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 827/1696] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:15.664752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:15.664937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:15.665035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 870/1697] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:15.666128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652cfc6-0239-48d4-8ddc-cd1ecbc720db None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:15.671767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:15.673785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:15.674072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:15.674354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:15.675106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:15.675416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:15.677804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:15.704998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 828/1698] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:43:15.710464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:15.724723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308f7f-b027-4201-a018-40dd7711a3f6 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:15.740539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:15.740539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:15.740539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:15.740539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:15.741019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:15.745247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:15.882676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 871/1699] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:15 2022] POST /identity/v3/projects => generated 403 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:15.888862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:15.904857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031d146-eaf9-48ce-b236-afb64f6dbcdc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:16.093441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:16.094404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:16.094693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:16.094999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:16.095665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:16.097609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:16.141401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:16.277118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 829/1700] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:15 2022] POST /identity/v3/users => generated 302 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:16.280527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:16.310620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eec90-fbe9-4281-b85d-53c5ca92fa96 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:16.321569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:16.322440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:16.323166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:16.323660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:16.325109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:16.327050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:16.340736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 872/1701] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:16.345209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:16.371486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb443dd-958a-4375-8ece-734a01bf1971 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:16.378685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:16.379375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:16.379659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:16.380074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:16.382125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:16.383741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:16.400050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 830/1702] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:16.401241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:16.417150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57378bcc-f2e1-475d-9aed-717a4039acd2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:16.428633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be46e57-8bb3-45a0-9b6b-b7bd0e61fc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:16.430663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be46e57-8bb3-45a0-9b6b-b7bd0e61fc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:16.431136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be46e57-8bb3-45a0-9b6b-b7bd0e61fc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:16.431557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be46e57-8bb3-45a0-9b6b-b7bd0e61fc4e admin admin] PATH_INFO: `/v3/projects/0a519e639afc43c39c0e53ad1ed0d2c9/users/8db7be148eac45daa545cf35f2694b42/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:16.484051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be46e57-8bb3-45a0-9b6b-b7bd0e61fc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a519e639afc43c39c0e53ad1ed0d2c9, user_id=8db7be148eac45daa545cf35f2694b42, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:16.486783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be46e57-8bb3-45a0-9b6b-b7bd0e61fc4e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:16.531774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:16.534327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:16.558652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42fc1-c305-40ea-b43c-233d388c773a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:16.570924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 873/1703] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:16 2022] PUT /identity/v3/projects/0a519e639afc43c39c0e53ad1ed0d2c9/users/8db7be148eac45daa545cf35f2694b42/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:16.576716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:16.578241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:16.578763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:16.579066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:16.579327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:16.579500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:16.580122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] PATH_INFO: `/v3/projects/7ef73b6dc97e48e096b9ce0ec84c4c89` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:16.583470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef73b6dc97e48e096b9ce0ec84c4c89)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:16.586743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:16.625904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:16.627186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:16.655504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:16.783988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 874/1704] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:16.845518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:16.846216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef73b6dc97e48e096b9ce0ec84c4c89', 'request_id': 'req-e41edd96-b480-4e7e-bd56-b69a471dbffc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:16.846968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:16.945403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef73b6dc97e48e096b9ce0ec84c4c89', 'request_id': 'req-e41edd96-b480-4e7e-bd56-b69a471dbffc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:16.953783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef73b6dc97e48e096b9ce0ec84c4c89', 'request_id': 'req-e41edd96-b480-4e7e-bd56-b69a471dbffc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:16.987658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ccf18-31c2-4c1f-ab66-21cf26a461f9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:16.994162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] The token cache is being invalidate because project 7ef73b6dc97e48e096b9ce0ec84c4c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:16.995809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:17.010632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e41edd96-b480-4e7e-bd56-b69a471dbffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:17.010632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 831/1705] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:16 2022] DELETE /identity/v3/projects/7ef73b6dc97e48e096b9ce0ec84c4c89 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:17.014539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375472ef-335f-4f01-90dd-3678c3867ded None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.015111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375472ef-335f-4f01-90dd-3678c3867ded None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.015338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375472ef-335f-4f01-90dd-3678c3867ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.059819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-375472ef-335f-4f01-90dd-3678c3867ded None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:17.061236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-375472ef-335f-4f01-90dd-3678c3867ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:17.108501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:17.110980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.111300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.111589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.113643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:17.226273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 832/1706] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:17.228824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.228824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.228998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.285011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:17.286252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:17.299746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:17.304080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:17.305448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 833/1707] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:17.310397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:17.335750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 875/1708] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:17.351381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0098c0-d43e-41ae-b2db-d35cd5b8c771 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:17.363125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:17.368305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.370129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.370857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.372405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:17.373065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:17.377611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:17.409681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 834/1709] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:43:17.416419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:17.443837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569d8a5-d5f6-4b66-9b27-e84d2836ef0f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:17.458335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:17.459549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.460068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.460445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.461191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:17.472275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:17.608091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 876/1710] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:17 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:17.609264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:17.639043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d81725-9fda-47ba-bc9d-44dcf43f32a5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:17.660307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:17.662403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:17.758525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:17.759321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.759616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.760615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.761387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:17.767099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:17.892013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 835/1711] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:17 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:17.902743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:17.906102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9971fd-cbc5-4c2c-980d-add0a23e76c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:17.922356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:17.923429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.923868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.924315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] PATH_INFO: `/v3/projects/012d42cfc6394ee2861cd66ad8be5a2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.928116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012d42cfc6394ee2861cd66ad8be5a2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:17.930621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:17.932857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e19f73-2d30-4e7e-bbdf-35387ef90c5c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:17.960761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:17.961490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:17.961776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:17.962081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:17.968016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:17.971742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:18.002750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 836/1712] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:18.007762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:18.022367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:18.033703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fa006-e1ce-4b5a-8775-2fbb138b82c7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:18.066743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:18.070226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.071556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.073230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.074091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:18.082575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:18.095512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012d42cfc6394ee2861cd66ad8be5a2c', 'request_id': 'req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:18.108163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012d42cfc6394ee2861cd66ad8be5a2c', 'request_id': 'req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:18.125897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 837/1713] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:18.127627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:18.164970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6274-b8cf-4cc9-b7a9-6408ea8adc3d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:18.219682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012d42cfc6394ee2861cd66ad8be5a2c', 'request_id': 'req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:18.235563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] The token cache is being invalidate because project 012d42cfc6394ee2861cd66ad8be5a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:18.243506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:18.245822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0cf5dff-9506-40e0-9dab-61a6aea9a2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:18.251610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 877/1714] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:17 2022] DELETE /identity/v3/projects/012d42cfc6394ee2861cd66ad8be5a2c => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:18.257183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9513cb3-3fbc-4473-b64a-bd1ab96f9cf7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.257735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9513cb3-3fbc-4473-b64a-bd1ab96f9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.258235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9513cb3-3fbc-4473-b64a-bd1ab96f9cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.308299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9513cb3-3fbc-4473-b64a-bd1ab96f9cf7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:18.312442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9513cb3-3fbc-4473-b64a-bd1ab96f9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:18.316405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803fc5e-84ac-438a-b431-e23d62a80431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:18.317432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803fc5e-84ac-438a-b431-e23d62a80431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.317716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803fc5e-84ac-438a-b431-e23d62a80431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.317989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803fc5e-84ac-438a-b431-e23d62a80431 admin admin] PATH_INFO: `/v3/projects/ddb5a2d57ed9469685f8227ccf7c9c40/users/06ca9ea3ad72413592acedc6c020afa6/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.341957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 878/1715] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:18.349801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.350347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.351073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.355049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803fc5e-84ac-438a-b431-e23d62a80431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb5a2d57ed9469685f8227ccf7c9c40, user_id=06ca9ea3ad72413592acedc6c020afa6, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:18.357865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803fc5e-84ac-438a-b431-e23d62a80431 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:18.392360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:18.395564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:18.396932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:18.426518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 879/1716] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:18.428224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:18.439841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b53637-062a-4e08-b7f1-aa6c4da3fedb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:18.448414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:18.459734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.460944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.462386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.463666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:18.465075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:18.484133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d6920-6ff6-4544-b5cd-596924a8db63 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:18.514828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 880/1717] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:18.516633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.517169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.517652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.524297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 838/1718] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:18 2022] PUT /identity/v3/projects/ddb5a2d57ed9469685f8227ccf7c9c40/users/06ca9ea3ad72413592acedc6c020afa6/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:18.526572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb98a8f-29dd-494c-9591-db871d36cba8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.532317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb98a8f-29dd-494c-9591-db871d36cba8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.534949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb98a8f-29dd-494c-9591-db871d36cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.576990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] MFA Rules not processed for user `46d0d621460b4e5b877ef421d3503340`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:18.576990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:18.620248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3cb98a8f-29dd-494c-9591-db871d36cba8 None None] MFA Rules not processed for user `8eb0ed48dd034be0ad63373e4a151cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:18.621673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3cb98a8f-29dd-494c-9591-db871d36cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:18.760196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 839/1719] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:18.762003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b9941-8b13-438e-8a8a-bed4a68bb102 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.762644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b9941-8b13-438e-8a8a-bed4a68bb102 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.763098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b9941-8b13-438e-8a8a-bed4a68bb102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.807972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-601b9941-8b13-438e-8a8a-bed4a68bb102 None None] MFA Rules not processed for user `df7a3101955b481f9a582168731ae76c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:18.809903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-601b9941-8b13-438e-8a8a-bed4a68bb102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:18.856076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 881/1720] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:18.857116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:18.918477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 840/1721] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:18.920916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f080cc9-167d-4567-80e4-c6faa3fe952e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:18.921597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f080cc9-167d-4567-80e4-c6faa3fe952e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:18.922039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f080cc9-167d-4567-80e4-c6faa3fe952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:18.963903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2f080cc9-167d-4567-80e4-c6faa3fe952e None None] MFA Rules not processed for user `06ca9ea3ad72413592acedc6c020afa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:18.968441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2f080cc9-167d-4567-80e4-c6faa3fe952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:19.029180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e07736-4e7e-4487-aa7a-0541c8af8aab None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:19.037871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:19.039119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:19.039431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:19.039720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:19.040444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:19.044781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:19.118040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 841/1722] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:19.118938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:19.119232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:19.119511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:19.175698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] MFA Rules not processed for user `8eb0ed48dd034be0ad63373e4a151cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:19.177221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:19.189543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 882/1723] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:18 2022] POST /identity/v3/projects => generated 393 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:19.190784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:19.199638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 842/1724] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:19.200417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:19.201258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:19.212247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:19.415422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4516cf7-48e1-439d-b6bd-7ad0918439a8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:19.455998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469179-c2bb-469c-9e17-6d6bdd6dde07 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:19.467790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:19.470225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:19.470686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:19.471119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:19.558802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:19.559908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:19.562013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:19.562480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:19.649066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:19.651179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:19.678498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 883/1725] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:19.691489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:19.699591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:19.731312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:19.732981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:19.766396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 843/1726] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:19.767554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:19.828844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3eae4-a9a9-4bda-8d9c-bcc90b676f1a service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:19.842976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:19.842976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:19.842976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:19.842976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:19.873920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:19.948095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff441e-1c05-4b64-933e-c9b60e7d3a95 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:19.965797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:19.967282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:19.967778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:19.968317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:19.970985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:19.978248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:20.003849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:20.005384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:20.048222 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 884/1727] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:20.082419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:20.084449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:20.093155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:20.111944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 844/1728] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:19 2022] POST /identity/v3/users => generated 297 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:20.114413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:20.125249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409eb73-30dd-4402-a935-b8238968d89d service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:20.141079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:20.142240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:20.142356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:20.142642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] PATH_INFO: `/v3/users/06ca9ea3ad72413592acedc6c020afa6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:20.147912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ca9ea3ad72413592acedc6c020afa6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:20.150056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:20.180871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712fcb-bbc1-4c93-a373-aad96708cfd6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:20.207521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:20.209631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:20.209922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:20.210372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:20.362858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:20.365012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:20.386226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 845/1729] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:20.387246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:20.417477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ca9ea3ad72413592acedc6c020afa6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:20.419036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d50f-776c-4ec3-9982-4634785730f5 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:20.430816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:20.431863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:20.436663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:20.437424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:20.439453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ca9ea3ad72413592acedc6c020afa6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:20.440809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:20.447227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:20.524778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 846/1730] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:20.529595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:20.540266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ca9ea3ad72413592acedc6c020afa6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:20.541024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ca9ea3ad72413592acedc6c020afa6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:20.541890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6741d0e7-2c1a-4d99-bd88-eeff76c3f279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ca9ea3ad72413592acedc6c020afa6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:20.553961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37261f6-104d-497d-985e-dd00286dd78c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:20.568842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:20.569688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:20.570115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:20.571044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:20.572611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:20.572861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:20.576809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 885/1731] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:20 2022] DELETE /identity/v3/users/06ca9ea3ad72413592acedc6c020afa6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:20.579255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:20.580627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:20.581052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:20.631912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 847/1732] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:20.634243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:20.636442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:20.662727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:20.669054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:20.684377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704b323-ad78-43fe-937b-70ab80d01cc3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:20.871468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:20.898976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aaa22-2c18-4dd1-adea-414b98ca4583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:20.900040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aaa22-2c18-4dd1-adea-414b98ca4583 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:20.900430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aaa22-2c18-4dd1-adea-414b98ca4583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:20.903402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 886/1733] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:20.903527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aaa22-2c18-4dd1-adea-414b98ca4583 admin admin] PATH_INFO: `/v3/projects/fd25f8dc07da4ce1aec5249f53544b1c/users/cf41993032cd44d787445cae723f3cf2/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:20.943712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aaa22-2c18-4dd1-adea-414b98ca4583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd25f8dc07da4ce1aec5249f53544b1c, user_id=cf41993032cd44d787445cae723f3cf2, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:20.950832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aaa22-2c18-4dd1-adea-414b98ca4583 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:21.007202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:21.010098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:21.068726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 848/1734] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:20 2022] PUT /identity/v3/projects/fd25f8dc07da4ce1aec5249f53544b1c/users/cf41993032cd44d787445cae723f3cf2/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:21.080632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5c8a-1eeb-452a-b401-37822a68de86 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:21.081235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5c8a-1eeb-452a-b401-37822a68de86 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:21.081563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5c8a-1eeb-452a-b401-37822a68de86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:21.179517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd862f1a-242e-4a14-bc4e-7ac38579a8a5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:21.218095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-feef5c8a-1eeb-452a-b401-37822a68de86 None None] MFA Rules not processed for user `cf41993032cd44d787445cae723f3cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:21.222755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-feef5c8a-1eeb-452a-b401-37822a68de86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:21.328506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:21.328506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:21.328506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:21.328506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:21.433650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 849/1735] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:21.437077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e152b-4520-4368-8d76-4cf5286469d4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:21.437498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e152b-4520-4368-8d76-4cf5286469d4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:21.439129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e152b-4520-4368-8d76-4cf5286469d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:21.606173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:21.608329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:21.629179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a02e152b-4520-4368-8d76-4cf5286469d4 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:21.637380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 887/1736] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:21.638163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:21.640803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a02e152b-4520-4368-8d76-4cf5286469d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:21.701568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f6c3f-4f44-4a1b-a88c-7143c026366e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:21.726887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:21.727588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:21.727881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:21.728510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:21.729243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:21.731140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:21.818292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 850/1737] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:21.820005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46054e0a-12d8-4cdd-9674-73ee2d60e82b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:21.820566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46054e0a-12d8-4cdd-9674-73ee2d60e82b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:21.821163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46054e0a-12d8-4cdd-9674-73ee2d60e82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:21.854944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 888/1738] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:21 2022] POST /identity/v3/projects => generated 393 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:21.859363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:21.883423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-46054e0a-12d8-4cdd-9674-73ee2d60e82b None None] MFA Rules not processed for user `17e0c6def385441bbdde18c63b7d99e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:21.884877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-46054e0a-12d8-4cdd-9674-73ee2d60e82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:22.028856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653678-87d5-44e5-8425-ef3ebef6a59e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.032520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 851/1739] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:22.038554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2feb37-4f62-476f-893b-8fb2d3e39c76 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.038990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2feb37-4f62-476f-893b-8fb2d3e39c76 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.039674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2feb37-4f62-476f-893b-8fb2d3e39c76 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.045298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 852/1740] 10.222.0.47 () {54 vars in 912 bytes} [Mon Jul 18 10:43:22 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:43:22.048178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.049546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.050202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.050770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.051398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.051757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.052105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.053191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.056614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.092178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:22.093596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:22.172769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 889/1741] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:21 2022] POST /identity/v3/users => generated 303 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:22.175175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.176823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 853/1742] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:43:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:22.189262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.191409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:22.225389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bef2d-11f8-4fd7-bbf7-c477d1ba37d1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.244704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.246222 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.248533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.249528 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.251124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.255476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.283557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 890/1743] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:22.289628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.325609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63c603-411c-4e2c-8597-b93fd9d1c81f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.341013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.345539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.346614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.347482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.377564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.383195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.397224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ef4b-aaf2-4525-a678-55056829a241 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.409693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 891/1744] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:22.410397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.442420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.442420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.442420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.442420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.445192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58bb32-858b-4dbf-8a83-06ffd4150973 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.457525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.458361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.458735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.459086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] PATH_INFO: `/v3/projects/b9e4687fbdc1454ea9b3e70e89506bb7/users/4d489ad13971410cbdc9857750f53493/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.470090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.483889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.508154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:22.513591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 854/1745] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:22.515692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.522622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:22.529732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e4687fbdc1454ea9b3e70e89506bb7, user_id=4d489ad13971410cbdc9857750f53493, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.533944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.662815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 892/1746] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:22 2022] PUT /identity/v3/projects/b9e4687fbdc1454ea9b3e70e89506bb7/users/4d489ad13971410cbdc9857750f53493/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:22.669494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.817389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfb5b-f9d7-4d10-95ff-5f3a3fbb5f61 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.831561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082c8ea-0d92-4cf6-aa2a-09ca0f4cfe75 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.836393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.837844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.838126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.839479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.840229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.843235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.872800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 893/1747] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:22.877993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:22.906760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2389-01cf-4c62-a062-4b81929ac81f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:22.917804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53865f88-a854-4ff9-9b41-72de3dac35cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.919599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53865f88-a854-4ff9-9b41-72de3dac35cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.920279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53865f88-a854-4ff9-9b41-72de3dac35cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.920279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53865f88-a854-4ff9-9b41-72de3dac35cc admin admin] PATH_INFO: `/v3/projects/b9e4687fbdc1454ea9b3e70e89506bb7/users/4d489ad13971410cbdc9857750f53493/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.925094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53865f88-a854-4ff9-9b41-72de3dac35cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e4687fbdc1454ea9b3e70e89506bb7, user_id=4d489ad13971410cbdc9857750f53493, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:22.928939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53865f88-a854-4ff9-9b41-72de3dac35cc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:22.957177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 894/1748] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:22 2022] PUT /identity/v3/projects/b9e4687fbdc1454ea9b3e70e89506bb7/users/4d489ad13971410cbdc9857750f53493/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:22.957322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:22.967074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.967425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.968278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:22.986612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:22.986612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:22.986946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:22.987197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:23.042118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] MFA Rules not processed for user `4d489ad13971410cbdc9857750f53493`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:23.043503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:23.117928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:23.119576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:23.144229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 855/1749] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:43:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:43:23.171868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 895/1750] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:23.189899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:23.191593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:23.344720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834a56b-4343-4bca-8035-5ba39cb2068c service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:23.574874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dfc6d-dd4e-4604-b4a1-353f5076361f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:23.575541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dfc6d-dd4e-4604-b4a1-353f5076361f service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:23.575831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dfc6d-dd4e-4604-b4a1-353f5076361f service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:23.576114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dfc6d-dd4e-4604-b4a1-353f5076361f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:23.652881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dfc6d-dd4e-4604-b4a1-353f5076361f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:23.653160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dfc6d-dd4e-4604-b4a1-353f5076361f service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:23.693751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 856/1751] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:23.792874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:23.794381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:23.821454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49855c-5c4e-4be0-9c81-e52a1ff148e3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:23.832785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:23.833420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:23.834283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:23.834558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:24.018087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:24.019564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:24.076167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 896/1752] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:24.634904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:24.635478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:24.635478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:24.770702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] MFA Rules not processed for user `8db7be148eac45daa545cf35f2694b42`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:24.772111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:24.845978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:25.233529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:25.233529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:25.234926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 857/1753] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:25.257995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2334-5c71-4e45-bf6f-e498cd561eef service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:25.263638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:25.265492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:25.265954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:25.266519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:25.267515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:25.270830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:25.451721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:25.665361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 897/1754] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:25 2022] POST /identity/v3/projects => generated 403 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:25.669491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:25.693850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad43364-7545-4a7a-9cfd-4ab757e2c110 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:25.973280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:25.974046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:25.974343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:25.974632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:25.982127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:25.984582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:26.049006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:26.066347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:26.088970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a681f-aa84-4963-aeba-b21e54da9e6d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:26.095549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:26.097389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:26.106174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:26.106174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] PATH_INFO: `/v3/projects/ddb5a2d57ed9469685f8227ccf7c9c40` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:26.106174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb5a2d57ed9469685f8227ccf7c9c40)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:26.106174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:26.205734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 858/1755] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:25 2022] POST /identity/v3/users => generated 308 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:26.210181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:26.238937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d5d1-80f5-4323-b6bc-fbde269db1fa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:26.275225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:26.276537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:26.285225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:26.285515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:26.287008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:26.291336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:26.324006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 859/1756] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:26.328219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:26.351818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26dbaf-adc5-4df7-a3b2-c0b5ae54533c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:26.407558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb5a2d57ed9469685f8227ccf7c9c40', 'request_id': 'req-a3098445-cf40-4dfa-aec6-a3651259f7c1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:26.460840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb5a2d57ed9469685f8227ccf7c9c40', 'request_id': 'req-a3098445-cf40-4dfa-aec6-a3651259f7c1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:26.618623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14fef1-f711-480f-8936-f5754c3e8c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:26.623826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14fef1-f711-480f-8936-f5754c3e8c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:26.624394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14fef1-f711-480f-8936-f5754c3e8c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:26.625444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14fef1-f711-480f-8936-f5754c3e8c95 admin admin] PATH_INFO: `/v3/projects/b9391467ef9a46c8add068f76f712197/users/e0de89f89a114a04a0e49a500f71a274/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:26.631134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb5a2d57ed9469685f8227ccf7c9c40', 'request_id': 'req-a3098445-cf40-4dfa-aec6-a3651259f7c1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:26.663360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] The token cache is being invalidate because project ddb5a2d57ed9469685f8227ccf7c9c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:26.667399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:26.671053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3098445-cf40-4dfa-aec6-a3651259f7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:26.684600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 898/1757] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:26 2022] DELETE /identity/v3/projects/ddb5a2d57ed9469685f8227ccf7c9c40 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:26.689822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf48704-fc27-459a-a899-5210fd7212c3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:26.691161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf48704-fc27-459a-a899-5210fd7212c3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:26.694115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf48704-fc27-459a-a899-5210fd7212c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:26.697667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14fef1-f711-480f-8936-f5754c3e8c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9391467ef9a46c8add068f76f712197, user_id=e0de89f89a114a04a0e49a500f71a274, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:26.705046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14fef1-f711-480f-8936-f5754c3e8c95 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:26.742018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdf48704-fc27-459a-a899-5210fd7212c3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:26.743154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdf48704-fc27-459a-a899-5210fd7212c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:26.753798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 899/1758] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:26.764135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:26.764883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:26.765610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:26.804876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 860/1759] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:26 2022] PUT /identity/v3/projects/b9391467ef9a46c8add068f76f712197/users/e0de89f89a114a04a0e49a500f71a274/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:26.807456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:26.807959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:26.808468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:26.819152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:26.821494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:26.840313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:26.844194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:26.983373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 900/1760] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:26.984407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:26.986211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:27.000025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04466cd2-22ae-4a5e-8abf-75d0cb5dfc74 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:27.020975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:27.023265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:27.023568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:27.024068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:27.025052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:27.026098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 861/1761] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Jul 18 10:43:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:27.027166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:27.029313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:27.043021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 901/1762] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:27.044471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:27.046633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:27.078275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262ac49-5d0a-4cfa-9b6d-d16a137ad4a8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:27.090077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545107-03d2-4e75-9219-3bb794570f33 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:27.096539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:27.097496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:27.097674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:27.098017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:27.098560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:27.099556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:27.099727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:27.099727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:27.104517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:27.105450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:27.112933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:27.132412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:27.134362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:27.150854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 902/1763] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:27.169946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 862/1764] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:27.169946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:27.186594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:27.241793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb846d2-78e3-4506-9843-c50a6d398424 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:27.264216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:27.272557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:27.273030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:27.273348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:27.276334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:27.281281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdf536-9a9d-48ce-9d2f-16db385d3a08 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:27.283393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:27.297191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:27.298479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:27.299018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:27.299698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] PATH_INFO: `/v3/projects/b9391467ef9a46c8add068f76f712197/users/e0de89f89a114a04a0e49a500f71a274/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:27.322973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9391467ef9a46c8add068f76f712197, user_id=e0de89f89a114a04a0e49a500f71a274, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:27.331577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:27.398425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 903/1765] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:27 2022] PUT /identity/v3/projects/b9391467ef9a46c8add068f76f712197/users/e0de89f89a114a04a0e49a500f71a274/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:27.473855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:27.490800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 863/1766] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:27 2022] POST /identity/v3/projects => generated 371 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:27.491782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:27.679581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760459fa-aae1-4536-911f-c866e6f008c0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:27.813999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:27.814862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:27.815164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:27.815454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:27.851790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:27.859525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:27.931035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:28.051563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:28.060122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 904/1767] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:27 2022] POST /identity/v3/users => generated 286 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:43:28.066577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fb7ab-dd0e-472e-bf29-2c6cfb2a1fcf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:28.081404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:28.081802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:28.082233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:28.082651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:28.084165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:28.086692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:28.119713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 864/1768] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:28.123976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:28.139095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47879ef7-0dd4-486d-80fb-45f4ed611d42 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:28.153378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dca67-d46d-4776-9fbc-ffdb02e99ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:28.154691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dca67-d46d-4776-9fbc-ffdb02e99ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:28.154997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dca67-d46d-4776-9fbc-ffdb02e99ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:28.155284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dca67-d46d-4776-9fbc-ffdb02e99ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:28.156230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dca67-d46d-4776-9fbc-ffdb02e99ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:28.158065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dca67-d46d-4776-9fbc-ffdb02e99ed1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:28.194365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:28.201184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:28.212169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 905/1769] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:28.226189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502a396-719f-45fd-92d7-af7a94fa29f2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:28.250659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:28.251698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:28.252071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:28.252421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] PATH_INFO: `/v3/projects/e78797b74b994de38384ab9c40a874c1/users/192352c2184f469aa7a4d633ea9028d2/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:28.312389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78797b74b994de38384ab9c40a874c1, user_id=192352c2184f469aa7a4d633ea9028d2, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:28.319633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:28.502677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 865/1770] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:28 2022] PUT /identity/v3/projects/e78797b74b994de38384ab9c40a874c1/users/192352c2184f469aa7a4d633ea9028d2/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:28.511875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:28.513768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:28.514412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:28.576359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] MFA Rules not processed for user `192352c2184f469aa7a4d633ea9028d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:28.579793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:28.865232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 906/1771] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:28.877867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:28.882924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:29.060995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007a511-53e4-4e3e-b316-42ea5483019d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:29.081785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b942b-7ab1-48b7-b414-febbf3087139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:29.082731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b942b-7ab1-48b7-b414-febbf3087139 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:29.083889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b942b-7ab1-48b7-b414-febbf3087139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:29.084178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b942b-7ab1-48b7-b414-febbf3087139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:29.127949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b942b-7ab1-48b7-b414-febbf3087139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:29.129246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b942b-7ab1-48b7-b414-febbf3087139 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:29.188863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 866/1772] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:29.226048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:29.459352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e133e-ccfb-4e40-b811-fe70333b4c63 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:29.475397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:29.476315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:29.476612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:29.480954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] PATH_INFO: `/v3/users/192352c2184f469aa7a4d633ea9028d2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:29.490320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192352c2184f469aa7a4d633ea9028d2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:29.490320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:29.769721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:29.815393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192352c2184f469aa7a4d633ea9028d2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:29.817269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192352c2184f469aa7a4d633ea9028d2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:29.905800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192352c2184f469aa7a4d633ea9028d2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:29.907410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192352c2184f469aa7a4d633ea9028d2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:29.908484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192352c2184f469aa7a4d633ea9028d2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:29.963518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 907/1773] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:29 2022] DELETE /identity/v3/users/192352c2184f469aa7a4d633ea9028d2 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:29.968521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758690a4-8720-4a14-b119-093b32b888d9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:29.968909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758690a4-8720-4a14-b119-093b32b888d9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:29.974579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758690a4-8720-4a14-b119-093b32b888d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:29.993866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:30.006051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-758690a4-8720-4a14-b119-093b32b888d9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:30.007613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-758690a4-8720-4a14-b119-093b32b888d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:30.144495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 867/1774] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:30.178997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:30.181248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:30.305810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9b166-b5f1-4133-8dbb-8a77f26312b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:30.318681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2c39c-ddaf-4351-aa3f-2f30fb1f101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:30.320715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c39c-ddaf-4351-aa3f-2f30fb1f101b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:30.321455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c39c-ddaf-4351-aa3f-2f30fb1f101b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:30.325558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c39c-ddaf-4351-aa3f-2f30fb1f101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:30.341841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2c39c-ddaf-4351-aa3f-2f30fb1f101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:30.344318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2c39c-ddaf-4351-aa3f-2f30fb1f101b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:30.361808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 908/1775] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:30.914956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554140f4-68e8-4a97-a7c7-62091791d2f1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:30.915624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554140f4-68e8-4a97-a7c7-62091791d2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:30.915925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554140f4-68e8-4a97-a7c7-62091791d2f1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:30.921604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 868/1776] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:43:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:43:30.998701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:30.998996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:30.999279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:31.053341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:31.058999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:31.248983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 909/1777] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:31.560271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554140f4-68e8-4a97-a7c7-62091791d2f1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:31.562357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-554140f4-68e8-4a97-a7c7-62091791d2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:31.606921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554140f4-68e8-4a97-a7c7-62091791d2f1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:31.623735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c84a32-b628-458e-ad78-109bfaef8486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:31.625668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c84a32-b628-458e-ad78-109bfaef8486 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:31.626059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c84a32-b628-458e-ad78-109bfaef8486 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:31.627747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c84a32-b628-458e-ad78-109bfaef8486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:31.661073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c84a32-b628-458e-ad78-109bfaef8486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:31.663010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c84a32-b628-458e-ad78-109bfaef8486 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:31.681702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 869/1778] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:32.281138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:32.328229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:32.330449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:32.345134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:32.375053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd6a3f-e043-43de-bba6-52680aeaf67a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:32.403683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:32.405534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:32.408841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:32.409464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] PATH_INFO: `/v3/users/cf41993032cd44d787445cae723f3cf2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:32.422100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf41993032cd44d787445cae723f3cf2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:32.425016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:32.652001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf41993032cd44d787445cae723f3cf2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:32.675414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf41993032cd44d787445cae723f3cf2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:32.764513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf41993032cd44d787445cae723f3cf2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:32.765561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf41993032cd44d787445cae723f3cf2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:32.766794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf41993032cd44d787445cae723f3cf2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:32.820101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 910/1779] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:32 2022] DELETE /identity/v3/users/cf41993032cd44d787445cae723f3cf2 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:32.825896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d98b7e-d8ec-4b4b-9e7f-f8646a401078 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:32.826991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d98b7e-d8ec-4b4b-9e7f-f8646a401078 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:32.827784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d98b7e-d8ec-4b4b-9e7f-f8646a401078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:32.861243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a7d98b7e-d8ec-4b4b-9e7f-f8646a401078 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:32.863097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a7d98b7e-d8ec-4b4b-9e7f-f8646a401078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:33.038521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 870/1780] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:33.055456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:33.157802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:33.160355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:33.317297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3c40f-e3ab-4e31-8cfc-c40212ef899f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:33.335578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867c941-f6d0-44a3-acd2-bcfd0406543f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:33.337798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867c941-f6d0-44a3-acd2-bcfd0406543f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:33.338110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867c941-f6d0-44a3-acd2-bcfd0406543f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:33.338395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867c941-f6d0-44a3-acd2-bcfd0406543f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:33.366759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867c941-f6d0-44a3-acd2-bcfd0406543f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:33.366759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867c941-f6d0-44a3-acd2-bcfd0406543f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:33.400831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 911/1781] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:33.556586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:34.014052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:34.141656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de28331-22ea-4eca-ac55-be7c31a44c37 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:34.142411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de28331-22ea-4eca-ac55-be7c31a44c37 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:34.143206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de28331-22ea-4eca-ac55-be7c31a44c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:34.210095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0de28331-22ea-4eca-ac55-be7c31a44c37 None None] MFA Rules not processed for user `17e0c6def385441bbdde18c63b7d99e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:34.211338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0de28331-22ea-4eca-ac55-be7c31a44c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:34.339633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 871/1782] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:34.877177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:35.012061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feed68-c4d4-488a-af68-5281ba3deec4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.013065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feed68-c4d4-488a-af68-5281ba3deec4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.013533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feed68-c4d4-488a-af68-5281ba3deec4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.015151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 912/1783] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:43:35.042526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.042819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.043101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.097662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:35.100406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:35.233633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50feed68-c4d4-488a-af68-5281ba3deec4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:35.235625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-50feed68-c4d4-488a-af68-5281ba3deec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:35.242024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 872/1784] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:43:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:35.271409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50feed68-c4d4-488a-af68-5281ba3deec4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:35.285252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:35.286217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.286781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.287214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] PATH_INFO: `/v3/projects/e78797b74b994de38384ab9c40a874c1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.292626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78797b74b994de38384ab9c40a874c1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:35.294839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:35.358665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:35.360525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:35.401764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793202a-e995-40c8-bb44-eaed545c51d2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:35.418431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f6284-7b3c-4891-9ce0-4f0dcfd38bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:35.422987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f6284-7b3c-4891-9ce0-4f0dcfd38bba service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.426605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f6284-7b3c-4891-9ce0-4f0dcfd38bba service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.426605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f6284-7b3c-4891-9ce0-4f0dcfd38bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.449348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f6284-7b3c-4891-9ce0-4f0dcfd38bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:35.457231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f6284-7b3c-4891-9ce0-4f0dcfd38bba service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:35.495030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 873/1785] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:35.499490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78797b74b994de38384ab9c40a874c1', 'request_id': 'req-4eb6c963-9054-4747-87f8-63059f5aa599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:35.500212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442f8f7-bc35-4ab5-b8dc-f19449089700 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.500939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442f8f7-bc35-4ab5-b8dc-f19449089700 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.501369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442f8f7-bc35-4ab5-b8dc-f19449089700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.519324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78797b74b994de38384ab9c40a874c1', 'request_id': 'req-4eb6c963-9054-4747-87f8-63059f5aa599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:35.565986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e442f8f7-bc35-4ab5-b8dc-f19449089700 None None] MFA Rules not processed for user `df7a3101955b481f9a582168731ae76c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:35.575177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e442f8f7-bc35-4ab5-b8dc-f19449089700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:35.665349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78797b74b994de38384ab9c40a874c1', 'request_id': 'req-4eb6c963-9054-4747-87f8-63059f5aa599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:35.711555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] The token cache is being invalidate because project e78797b74b994de38384ab9c40a874c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:35.711876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:35.713962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4eb6c963-9054-4747-87f8-63059f5aa599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:35.719572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 913/1786] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:35 2022] DELETE /identity/v3/projects/e78797b74b994de38384ab9c40a874c1 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:35.725001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.725300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.727925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.796009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:35.804297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:35.818264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 874/1787] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:35.818264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7b12-2a8e-4869-9522-722bcf94a8ee None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:35.818264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7b12-2a8e-4869-9522-722bcf94a8ee None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:35.818264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7b12-2a8e-4869-9522-722bcf94a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:35.847139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:35.856322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e4dd7b12-2a8e-4869-9522-722bcf94a8ee None None] MFA Rules not processed for user `46d0d621460b4e5b877ef421d3503340`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:35.857681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e4dd7b12-2a8e-4869-9522-722bcf94a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:35.959877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 875/1788] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:36.000775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 914/1789] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:36.009610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.009930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.010169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.032627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:36.034137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:36.041522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:36.054625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 876/1790] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:36.060618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.079880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0c04a-9e7b-4f89-9271-a502e23ef21f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.091354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.099111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.099111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.099111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.099111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:36.099111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.099111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:36.124746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 915/1791] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:36.130556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.148932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44279e27-c420-48bb-a982-7ac77147c508 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.160574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.162906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.163751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.164243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.165256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.167706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:36.293985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 877/1792] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:36 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:36.297931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.336270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21118-10cb-4713-812d-01705896db04 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.453020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.453721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.454195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.454520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.455470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.458519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:36.564861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 916/1793] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:36 2022] POST /identity/v3/users => generated 287 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:36.570752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.586996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371061-0182-49d0-9a71-e1d10d0c11f8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.598521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.604196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.604822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.605268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.607234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.610979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:36.679737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 878/1794] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:36.684371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.712431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48185e9b-8baf-4f22-b2e3-7926b5513dff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.757629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.759818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.760174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.762789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.763679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.765448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:36.823838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 917/1795] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:36.829761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.852444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:36.853306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa557e-8272-4df7-b053-e42e54562cba admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.858024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:36.878304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6467e-d749-4cc8-b5d2-4132cb0a60a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.878304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6467e-d749-4cc8-b5d2-4132cb0a60a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.878304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6467e-d749-4cc8-b5d2-4132cb0a60a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.878304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6467e-d749-4cc8-b5d2-4132cb0a60a1 admin admin] PATH_INFO: `/v3/projects/23816009bf9e4451beb6caf071355167/users/9621f3b35abb4419a5b3f0d0f2780bdf/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.917332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16522c7-c028-4fc6-ac40-7a117adb4125 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:36.939879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:36.940814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:36.941109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:36.941394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] PATH_INFO: `/v3/projects/fd25f8dc07da4ce1aec5249f53544b1c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:36.951458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd25f8dc07da4ce1aec5249f53544b1c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.961925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:36.967057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6467e-d749-4cc8-b5d2-4132cb0a60a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23816009bf9e4451beb6caf071355167, user_id=9621f3b35abb4419a5b3f0d0f2780bdf, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:36.970836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6467e-d749-4cc8-b5d2-4132cb0a60a1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:37.119362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 879/1796] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:36 2022] PUT /identity/v3/projects/23816009bf9e4451beb6caf071355167/users/9621f3b35abb4419a5b3f0d0f2780bdf/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:37.129638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:37.130918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:37.131625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:37.159637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:37.182187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] MFA Rules not processed for user `9621f3b35abb4419a5b3f0d0f2780bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:37.183767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:37.257563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:37.315825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd25f8dc07da4ce1aec5249f53544b1c', 'request_id': 'req-446eb59e-27d6-4d00-8d59-f6e01295131f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:37.336577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd25f8dc07da4ce1aec5249f53544b1c', 'request_id': 'req-446eb59e-27d6-4d00-8d59-f6e01295131f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:37.412182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 880/1797] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:37.417092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:37.490710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd25f8dc07da4ce1aec5249f53544b1c', 'request_id': 'req-446eb59e-27d6-4d00-8d59-f6e01295131f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:37.507794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] The token cache is being invalidate because project fd25f8dc07da4ce1aec5249f53544b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:37.508100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:37.516078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:37.521828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 918/1798] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:36 2022] DELETE /identity/v3/projects/fd25f8dc07da4ce1aec5249f53544b1c => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:37.526190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:37.532254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:37.597816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07184b-b3c8-4eb0-a437-628cfbfe4f58 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:37.722982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb59e-27d6-4d00-8d59-f6e01295131f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:37.736522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:37.742707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:37.742707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:37.745325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] PATH_INFO: `/v3/users/4d489ad13971410cbdc9857750f53493` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:37.758977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d489ad13971410cbdc9857750f53493)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:37.768439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:37.769351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:37.769567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:37.769754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:37.772658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:37.775489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:37.775637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:37.960430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 881/1799] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:37 2022] POST /identity/v3/projects => generated 375 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:37.964514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:37.967613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:38.077404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244c7de-c325-4d7b-a830-2e6675d6b445 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:38.087208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:38.088740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:38.089050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:38.089328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:38.090242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:38.099916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:38.130859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d489ad13971410cbdc9857750f53493'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:38.153073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d489ad13971410cbdc9857750f53493'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:38.256388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 882/1800] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:37 2022] POST /identity/v3/users => generated 294 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:43:38.257891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:38.277273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16642b-6be7-4f36-a234-52c77c70a860 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:38.280202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d489ad13971410cbdc9857750f53493'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:38.281867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d489ad13971410cbdc9857750f53493'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:38.289263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0c03c2b-ab50-42da-80b9-21b92f7900b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d489ad13971410cbdc9857750f53493'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:38.292928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:38.294436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:38.294774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:38.295056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:38.298071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:38.299761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:38.343342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 883/1801] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:38.346841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:38.351922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:38.367373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 919/1802] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:37 2022] DELETE /identity/v3/users/4d489ad13971410cbdc9857750f53493 => generated 0 bytes in 842 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:38.371242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d789b-0732-4ecb-85d3-905889bd2037 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:38.413046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:38.413432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:38.413787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:38.534073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:38.535043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:38.535271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:38.535569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] PATH_INFO: `/v3/projects/212e555b56974e1bac17de03406bcf6c/users/374c278b6498465b8072a2a928a9e791/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:38.556595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] MFA Rules not processed for user `e0de89f89a114a04a0e49a500f71a274`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:38.558962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:38.578718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212e555b56974e1bac17de03406bcf6c, user_id=374c278b6498465b8072a2a928a9e791, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:38.597908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:38.754474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 884/1803] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:38 2022] PUT /identity/v3/projects/212e555b56974e1bac17de03406bcf6c/users/374c278b6498465b8072a2a928a9e791/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:38.758723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 920/1804] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:38.770838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:39.009820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62f83d-ec56-4489-955c-2e8496b81345 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:39.022515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b03c64-034f-48d6-94ea-f8d4bb3d8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:39.023208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03c64-034f-48d6-94ea-f8d4bb3d8c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:39.023497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03c64-034f-48d6-94ea-f8d4bb3d8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:39.024399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03c64-034f-48d6-94ea-f8d4bb3d8c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:39.029841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b03c64-034f-48d6-94ea-f8d4bb3d8c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:39.031574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b03c64-034f-48d6-94ea-f8d4bb3d8c4e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:39.071565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 885/1805] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:39.075750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:39.108570 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29dc95-e6d0-4fe0-a9da-d6c91a4040e7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:39.127107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:39.134681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:39.134681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:39.134681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] PATH_INFO: `/v3/projects/212e555b56974e1bac17de03406bcf6c/users/374c278b6498465b8072a2a928a9e791/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:39.134681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] RBAC: Authorizing `identity:create_grant(project_id=212e555b56974e1bac17de03406bcf6c, user_id=374c278b6498465b8072a2a928a9e791, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:39.143245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:39.184017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 921/1806] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:39 2022] PUT /identity/v3/projects/212e555b56974e1bac17de03406bcf6c/users/374c278b6498465b8072a2a928a9e791/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:39.199009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:39.199318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:39.199603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:39.291327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] MFA Rules not processed for user `374c278b6498465b8072a2a928a9e791`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:39.293235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:39.438668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 886/1807] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:39.450962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:39.454203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:39.613411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef889ee-105f-4a46-b061-c78b56a6387c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:39.631649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:39.632795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:39.633088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:39.633586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:39.636654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:39.720081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:39.721770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4ebbd-62cc-4310-9f5d-b23f77669283 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:39.732740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 922/1808] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:39.991032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:39.993101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:40.134396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f3ae6-2367-4dde-998a-d526dd9515dc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:40.141145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:40.142041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.142583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.142872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] PATH_INFO: `/v3/projects/b9e4687fbdc1454ea9b3e70e89506bb7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.173847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e4687fbdc1454ea9b3e70e89506bb7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:40.177013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:40.409368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e4687fbdc1454ea9b3e70e89506bb7', 'request_id': 'req-2e736cd9-73f3-49b1-bff3-951fe73ff542'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:40.552833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:40.554152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e4687fbdc1454ea9b3e70e89506bb7', 'request_id': 'req-2e736cd9-73f3-49b1-bff3-951fe73ff542'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:40.560116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e4687fbdc1454ea9b3e70e89506bb7', 'request_id': 'req-2e736cd9-73f3-49b1-bff3-951fe73ff542'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:40.571670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] The token cache is being invalidate because project b9e4687fbdc1454ea9b3e70e89506bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:40.571965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:40.572634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2e736cd9-73f3-49b1-bff3-951fe73ff542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:40.574255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 887/1809] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:39 2022] DELETE /identity/v3/projects/b9e4687fbdc1454ea9b3e70e89506bb7 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:40.579698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.580423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.580555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.600114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:40.601374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:40.673101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 923/1810] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:40.681335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a692747-362b-4fda-9fc1-d84ab0c31ea7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.691383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a692747-362b-4fda-9fc1-d84ab0c31ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.691383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a692747-362b-4fda-9fc1-d84ab0c31ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.700905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3a692747-362b-4fda-9fc1-d84ab0c31ea7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:40.702189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3a692747-362b-4fda-9fc1-d84ab0c31ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:40.717308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 888/1811] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:40.724530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:40.735477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098001d9-d114-4090-b3cd-99303b77bc50 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:40.741413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:40.742081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.742366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.743470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.744596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:40.745303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:40.747658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b1497-8bc8-44f1-95a7-8bef1dd4ec05 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:40.758234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc778b7-8f04-4420-9e39-84cd3ba8c48d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.758615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc778b7-8f04-4420-9e39-84cd3ba8c48d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.758928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc778b7-8f04-4420-9e39-84cd3ba8c48d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.760402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 889/1812] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:43:40.764567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 924/1813] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:40.765609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc778b7-8f04-4420-9e39-84cd3ba8c48d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:40.783080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc778b7-8f04-4420-9e39-84cd3ba8c48d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:40.819597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:40.819597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.819597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.819597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.819878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:40.821825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:40.822156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:40.822637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:40.823099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:40.852158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:40.853505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:40.937213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 925/1814] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:43:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:40.948440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:40.949962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:40.967606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf345661-9b1f-47ed-bd03-c463123d9c86 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:40.967842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 890/1815] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:40 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:40.971630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.045090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.046324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.046620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.046903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.056839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0403ba-763c-431f-8cd1-a441c3087a3c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.057357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.058869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.063367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.064193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.071654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.071654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.071654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.071654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.071971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 926/1816] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:41.213529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 891/1817] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:40 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:43:41.215282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.226070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b8c23-060e-4337-8012-599eaa42c08d service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.230732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.237632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.237632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.237632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.237632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.237632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.244214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 927/1818] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:41.248309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.263733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cfa09-5f33-4ab2-860a-ebeb75f9e52d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.270119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af77736-34d0-46cb-9996-8a0f0912630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.270119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af77736-34d0-46cb-9996-8a0f0912630f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.270119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af77736-34d0-46cb-9996-8a0f0912630f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.270119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af77736-34d0-46cb-9996-8a0f0912630f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.271245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af77736-34d0-46cb-9996-8a0f0912630f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.273105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af77736-34d0-46cb-9996-8a0f0912630f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.283511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 892/1819] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:41.287984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.299397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb401014-47a1-48c9-9f2e-3ac96c39b260 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.308100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.308919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.309209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.309478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] PATH_INFO: `/v3/projects/43e7da9a84b543d28ab812ff811b54a7/users/b8740f74940247959c39f130145a5338/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.326033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e7da9a84b543d28ab812ff811b54a7, user_id=b8740f74940247959c39f130145a5338, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.329033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.439609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 928/1820] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:41 2022] PUT /identity/v3/projects/43e7da9a84b543d28ab812ff811b54a7/users/b8740f74940247959c39f130145a5338/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:41.452125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.452403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.452705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.482559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] MFA Rules not processed for user `b8740f74940247959c39f130145a5338`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:41.483876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:41.552393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 893/1821] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:41.557324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.626330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c99279-5cae-42f1-a8af-e9e631436bca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.631700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.632356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.632640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.632968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.633683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.635578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.756893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 929/1822] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:41 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:41.758889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.769838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f021abb-ca45-45de-bc0b-321d2a001f49 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.827231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:41.827856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.828539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.828857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.829140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.831910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.832131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:41.954532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 894/1823] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:41 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:41.954732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:41.973242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15863707-c362-4209-8e78-ea2184cdea73 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:41.985699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:41.987105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:41.988745 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:41.989132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:41.995021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:41.995021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:42.013229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 930/1824] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:42.024997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:42.036535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24232084-08b1-4010-a05c-6e86ffd696c7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:42.045238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:42.046215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:42.046582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:42.046936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] PATH_INFO: `/v3/projects/e66974f031994024a74dff17a449de5b/users/eeca02a905c7461e8c32d02b31e8f3f3/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:42.069922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66974f031994024a74dff17a449de5b, user_id=eeca02a905c7461e8c32d02b31e8f3f3, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:42.072763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:42.135671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:42.137359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:42.151970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7e580-808c-4997-8b93-df36df404666 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:42.156632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:42.157543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:42.157832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:42.158111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] PATH_INFO: `/v3/users/8db7be148eac45daa545cf35f2694b42` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:42.159370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db7be148eac45daa545cf35f2694b42)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:42.161563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:42.242867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 895/1825] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:42 2022] PUT /identity/v3/projects/e66974f031994024a74dff17a449de5b/users/eeca02a905c7461e8c32d02b31e8f3f3/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:42.246904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:42.308992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20495f12-5d4f-46f8-b316-999f05b67eb2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:42.313585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:42.314248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:42.314533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:42.314809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:42.315530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:42.317892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:42.327949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 896/1826] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:42.332353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:42.342007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ec766-3ec4-4193-b62e-240d7c67f830 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:42.346609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022ccb-c7ae-4fdb-bb4d-657d0b2d37fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:42.348903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022ccb-c7ae-4fdb-bb4d-657d0b2d37fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:42.348903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022ccb-c7ae-4fdb-bb4d-657d0b2d37fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:42.348903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022ccb-c7ae-4fdb-bb4d-657d0b2d37fd admin admin] PATH_INFO: `/v3/projects/e66974f031994024a74dff17a449de5b/users/eeca02a905c7461e8c32d02b31e8f3f3/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:42.350377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e022ccb-c7ae-4fdb-bb4d-657d0b2d37fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66974f031994024a74dff17a449de5b, user_id=eeca02a905c7461e8c32d02b31e8f3f3, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:42.353141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e022ccb-c7ae-4fdb-bb4d-657d0b2d37fd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:42.363200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 897/1827] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:42 2022] PUT /identity/v3/projects/e66974f031994024a74dff17a449de5b/users/eeca02a905c7461e8c32d02b31e8f3f3/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:42.373733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:42.377585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:42.377585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:42.401983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] MFA Rules not processed for user `eeca02a905c7461e8c32d02b31e8f3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:42.403243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:42.469110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 898/1828] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:42.480991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:42.482820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:42.516672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db7be148eac45daa545cf35f2694b42'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:42.531554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db7be148eac45daa545cf35f2694b42'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:42.540816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d91cb-7aba-4fb3-9db0-7603dc8569eb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:42.544442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:42.545120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:42.545406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:42.545686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:42.556023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:42.557635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:42.566975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 899/1829] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:42.610278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db7be148eac45daa545cf35f2694b42'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:42.622756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db7be148eac45daa545cf35f2694b42'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:42.622756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db7be148eac45daa545cf35f2694b42'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:42.637981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 931/1830] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:42 2022] DELETE /identity/v3/users/8db7be148eac45daa545cf35f2694b42 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:42.806369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:43.068825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:43.215053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d65e4b-5815-47f5-88d9-a4335c55b017 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:43.313116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:43.314613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:43.315202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:43.315639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] PATH_INFO: `/v3/projects/0a519e639afc43c39c0e53ad1ed0d2c9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:43.321926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a519e639afc43c39c0e53ad1ed0d2c9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:43.324256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:43.450925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a519e639afc43c39c0e53ad1ed0d2c9', 'request_id': 'req-ed4942b6-083f-4092-9d35-705164c18634'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.544554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a519e639afc43c39c0e53ad1ed0d2c9', 'request_id': 'req-ed4942b6-083f-4092-9d35-705164c18634'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.553053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a519e639afc43c39c0e53ad1ed0d2c9', 'request_id': 'req-ed4942b6-083f-4092-9d35-705164c18634'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.570644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:43.573465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:43.573611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] The token cache is being invalidate because project 0a519e639afc43c39c0e53ad1ed0d2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:43.573611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.573611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.579279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 900/1831] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:43 2022] DELETE /identity/v3/projects/0a519e639afc43c39c0e53ad1ed0d2c9 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:43.580773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:43.582340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:43.593389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031366-1185-4f81-b517-ed1af29f286e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:43.652454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:43.653380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:43.653674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:43.654263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4942b6-083f-4092-9d35-705164c18634 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:43.654395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] PATH_INFO: `/v3/users/b8740f74940247959c39f130145a5338` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:43.655492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8740f74940247959c39f130145a5338)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:43.657680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:43.658940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:43.659813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:43.660087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:43.660367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] PATH_INFO: `/v3/users/9621f3b35abb4419a5b3f0d0f2780bdf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:43.662290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9621f3b35abb4419a5b3f0d0f2780bdf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:43.669000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:43.858713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8740f74940247959c39f130145a5338'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.875552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8740f74940247959c39f130145a5338'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.884103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9621f3b35abb4419a5b3f0d0f2780bdf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.897183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9621f3b35abb4419a5b3f0d0f2780bdf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:43.905442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9621f3b35abb4419a5b3f0d0f2780bdf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.027407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8740f74940247959c39f130145a5338'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.028114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8740f74940247959c39f130145a5338'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.028879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8740f74940247959c39f130145a5338'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.052868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 932/1832] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:43 2022] DELETE /identity/v3/users/b8740f74940247959c39f130145a5338 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:44.053810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:44.055188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:44.141139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b74a5-8b94-4238-8062-f53c5a2909a2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:44.157247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:44.158070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:44.158355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:44.158629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] PATH_INFO: `/v3/users/e0de89f89a114a04a0e49a500f71a274` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:44.159858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0de89f89a114a04a0e49a500f71a274)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:44.171554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:44.181369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9621f3b35abb4419a5b3f0d0f2780bdf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.182002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3410bb64-6d7b-44c0-8653-aa2590a732e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9621f3b35abb4419a5b3f0d0f2780bdf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.189440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 901/1833] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:43 2022] DELETE /identity/v3/users/9621f3b35abb4419a5b3f0d0f2780bdf => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:44.190792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d85e43-e04b-4dcd-8d63-44ca5d559f3f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:44.191126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d85e43-e04b-4dcd-8d63-44ca5d559f3f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:44.191421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d85e43-e04b-4dcd-8d63-44ca5d559f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:44.227821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-27d85e43-e04b-4dcd-8d63-44ca5d559f3f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:44.230014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-27d85e43-e04b-4dcd-8d63-44ca5d559f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:44.335006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 902/1834] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:44.336640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:44.337116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:44.337540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:44.347995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:44.385052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:44.385052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:44.389870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 903/1835] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:44.391145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:44.392318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:44.480660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:44.529035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0de89f89a114a04a0e49a500f71a274'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.545727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bdf97-d408-4396-8e1d-2b929a2cbcfe None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:44.563677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:44.566822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:44.567356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:44.567855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:44.568838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0de89f89a114a04a0e49a500f71a274'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.603258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:44.607773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:44.630726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 904/1836] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:44.638407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:44.653454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44784-065a-4c94-ad9a-e367bf59f4ad service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:44.667052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:44.668185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:44.668528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:44.668864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:44.684219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:44.686375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:44.706833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 905/1837] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:44.707271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0de89f89a114a04a0e49a500f71a274'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.708394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0de89f89a114a04a0e49a500f71a274'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.716517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17409649-ac3e-4ab1-bf2e-12d2d6b1f01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0de89f89a114a04a0e49a500f71a274'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:44.727825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 933/1838] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:44 2022] DELETE /identity/v3/users/e0de89f89a114a04a0e49a500f71a274 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:46.012163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:46.013853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:46.027673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe76e05-335d-4973-aaf8-4676ac947362 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:46.094501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:46.095862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.100186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.100186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] PATH_INFO: `/v3/projects/b9391467ef9a46c8add068f76f712197` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.100186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9391467ef9a46c8add068f76f712197)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:46.101108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:46.463030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9391467ef9a46c8add068f76f712197', 'request_id': 'req-2510d6ce-cab2-402c-93f3-235c5df86aed'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:46.579274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9391467ef9a46c8add068f76f712197', 'request_id': 'req-2510d6ce-cab2-402c-93f3-235c5df86aed'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:46.588162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9391467ef9a46c8add068f76f712197', 'request_id': 'req-2510d6ce-cab2-402c-93f3-235c5df86aed'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:46.614959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] The token cache is being invalidate because project b9391467ef9a46c8add068f76f712197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:46.615346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:46.616057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2510d6ce-cab2-402c-93f3-235c5df86aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:46.618679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 906/1839] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:46 2022] DELETE /identity/v3/projects/b9391467ef9a46c8add068f76f712197 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:46.628014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4793e69-11c8-438b-9606-a5422af0fd18 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.628874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4793e69-11c8-438b-9606-a5422af0fd18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.629464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4793e69-11c8-438b-9606-a5422af0fd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.649989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a4793e69-11c8-438b-9606-a5422af0fd18 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:46.651516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a4793e69-11c8-438b-9606-a5422af0fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:46.687180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d73b7c-1bdc-48af-8116-98590f0a94bc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.688098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d73b7c-1bdc-48af-8116-98590f0a94bc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.688570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d73b7c-1bdc-48af-8116-98590f0a94bc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.690272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 907/1840] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:43:46.712176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.713146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.713628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.737624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:46.739662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:46.801770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 934/1841] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:46.813836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.814406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.814840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.848359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:46.851254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:46.883733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 935/1842] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:46.884928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:46.886833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:46.892865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 908/1843] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:43:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:46.893512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:46.910064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54770b6c-cc82-474d-9f11-da9cb22412a9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:46.913789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3e4c-f80b-41dd-80ad-35491b0c970e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:46.923851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:46.924188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:46.930476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.930476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.930476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.930762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:46.930762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:46.930762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] PATH_INFO: `/v3/projects/43e7da9a84b543d28ab812ff811b54a7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:46.931019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:46.931101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e7da9a84b543d28ab812ff811b54a7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:46.931503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:46.934042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:46.935905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:47.002563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 909/1844] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:47.002563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:47.002563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7fd7-3928-46e4-8f72-51c045b0e35c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:47.011670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:47.012366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:47.012948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:47.012948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:47.013664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:47.015595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:47.119158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e7da9a84b543d28ab812ff811b54a7', 'request_id': 'req-a1bfff77-7bc9-4509-b289-67da29fa009f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.146956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e7da9a84b543d28ab812ff811b54a7', 'request_id': 'req-a1bfff77-7bc9-4509-b289-67da29fa009f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.172903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 910/1845] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:46 2022] POST /identity/v3/projects => generated 363 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:47.174368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:47.179975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:47.247653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e7da9a84b543d28ab812ff811b54a7', 'request_id': 'req-a1bfff77-7bc9-4509-b289-67da29fa009f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.268358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] The token cache is being invalidate because project 43e7da9a84b543d28ab812ff811b54a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:47.268658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.273781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.275576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29550d6-fd5d-445a-a973-6c8b927fea0a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:47.276751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 936/1846] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:46 2022] DELETE /identity/v3/projects/43e7da9a84b543d28ab812ff811b54a7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:47.277339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:47.279039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:47.281035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:47.282102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:47.282665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:47.283204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] PATH_INFO: `/v3/projects/23816009bf9e4451beb6caf071355167` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:47.285014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=23816009bf9e4451beb6caf071355167)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:47.287824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:47.444486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfff77-7bc9-4509-b289-67da29fa009f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:47.523036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23816009bf9e4451beb6caf071355167', 'request_id': 'req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.536408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:47.536821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:47.537114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:47.537446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:47.539641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:47.576065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:47.577753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:47.592265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 937/1847] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:47.593411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:47.594866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:47.622902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b028-47af-4b16-835c-51e862aa2e5e service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:47.666706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23816009bf9e4451beb6caf071355167', 'request_id': 'req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.677918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23816009bf9e4451beb6caf071355167', 'request_id': 'req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.701910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] The token cache is being invalidate because project 23816009bf9e4451beb6caf071355167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:47.702507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.703526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:47.705438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:47.706128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:47.706417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:47.706698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:47.706983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 911/1848] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:47 2022] DELETE /identity/v3/projects/23816009bf9e4451beb6caf071355167 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:47.710254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:47.710517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:47.711819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:47.713585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:47.732525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1b8a-1c7f-4353-bf38-ff1b29492c6a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:47.744230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:47.746742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:47.747644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:47.748161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] PATH_INFO: `/v3/users/eeca02a905c7461e8c32d02b31e8f3f3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:47.749913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeca02a905c7461e8c32d02b31e8f3f3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:47.752921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:47.912478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 938/1849] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:47 2022] POST /identity/v3/users => generated 282 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:47.912478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:47.915970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:47.934335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfca7c-9874-4c98-9494-166bf201f399 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:47.942529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:47.943892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:47.944310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:47.944735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] PATH_INFO: `/v3/users/374c278b6498465b8072a2a928a9e791` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:47.951469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=374c278b6498465b8072a2a928a9e791)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:47.953896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:48.115772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeca02a905c7461e8c32d02b31e8f3f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.142490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeca02a905c7461e8c32d02b31e8f3f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.160525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeca02a905c7461e8c32d02b31e8f3f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.225314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '374c278b6498465b8072a2a928a9e791'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.236111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '374c278b6498465b8072a2a928a9e791'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.274382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeca02a905c7461e8c32d02b31e8f3f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.274382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeca02a905c7461e8c32d02b31e8f3f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.278867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 912/1850] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:47 2022] DELETE /identity/v3/users/eeca02a905c7461e8c32d02b31e8f3f3 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:48.280319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:48.284700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:48.362118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '374c278b6498465b8072a2a928a9e791'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.364592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '374c278b6498465b8072a2a928a9e791'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.366334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '374c278b6498465b8072a2a928a9e791'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:48.389840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ddda-b835-4e4d-af19-b73cffa22e0c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:48.396719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:48.397527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:48.397953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:48.398334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:48.399353 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:48.401290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:48.413931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 939/1851] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:47 2022] DELETE /identity/v3/users/374c278b6498465b8072a2a928a9e791 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:48.419844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 913/1852] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:48.420918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:48.425567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:48.598616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193de57-8c26-4a4b-a512-800a4154d2be admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:48.605791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883042bb-4c21-4d6c-9823-da64c61e2c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:48.606502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883042bb-4c21-4d6c-9823-da64c61e2c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:48.606795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883042bb-4c21-4d6c-9823-da64c61e2c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:48.608192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883042bb-4c21-4d6c-9823-da64c61e2c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:48.609511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883042bb-4c21-4d6c-9823-da64c61e2c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:48.611639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883042bb-4c21-4d6c-9823-da64c61e2c7e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:48.642438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 940/1853] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:48.643833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:48.673198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228b4c-cb33-4477-bfd7-984af2c93b9a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:48.681661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:48.683602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:48.683992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:48.684405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] PATH_INFO: `/v3/projects/712b0a9750b5472e8884f08800d662eb/users/79cdbc9918c640b08d8c7a1ea5f688d4/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:48.713193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] RBAC: Authorizing `identity:create_grant(project_id=712b0a9750b5472e8884f08800d662eb, user_id=79cdbc9918c640b08d8c7a1ea5f688d4, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:48.717613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:48.837334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 914/1854] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:48 2022] PUT /identity/v3/projects/712b0a9750b5472e8884f08800d662eb/users/79cdbc9918c640b08d8c7a1ea5f688d4/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:43:48.849173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:48.849987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:48.850420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:48.881313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] MFA Rules not processed for user `79cdbc9918c640b08d8c7a1ea5f688d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:48.882728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:48.993293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 941/1855] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:48.998007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.091275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14199ff-d697-4ee0-9759-3ec47f5efcae admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.098819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.100022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.100162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.100427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.101245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.103964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.229610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:49.275670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 915/1856] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:48 2022] POST /identity/v3/projects => generated 363 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:49.278645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.294320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317dcb4-dfed-4f56-9270-a522dd28f5d0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.388068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.388856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.389158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.389618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.390328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.392263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.413388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:49.489618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 942/1857] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:49 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:49.494442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.514187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185de7-19ad-4063-8f11-c0d9ece3becc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.520730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.522630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.523095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.523374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.524131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.526877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.540731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 916/1858] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:49.545089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.556893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f403ad6-db68-4302-875a-5ee25065e391 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.561879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.562808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.563101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.563394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] PATH_INFO: `/v3/projects/32c1183ac6904a0db4cb631c1a27b7dc/users/a98556109cf34a7fa1c4bef5d608fafc/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.579455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c1183ac6904a0db4cb631c1a27b7dc, user_id=a98556109cf34a7fa1c4bef5d608fafc, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.587534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.688766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 943/1859] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:49 2022] PUT /identity/v3/projects/32c1183ac6904a0db4cb631c1a27b7dc/users/a98556109cf34a7fa1c4bef5d608fafc/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:49.694141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.713934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.715684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:49.728258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0891d2-8cdc-492d-a6d9-c5ba9a296b3b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.802524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c009b26-4bb4-473b-a9bd-b56cf59250f0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.811687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.812725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.814725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.815820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] PATH_INFO: `/v3/projects/212e555b56974e1bac17de03406bcf6c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.825533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212e555b56974e1bac17de03406bcf6c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.825533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.853916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.854665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.857204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.857652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.858830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.865216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.871891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 917/1860] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:49.871891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:49.871891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:49.888131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37077aa9-67a5-4423-a62b-477d1a551f75 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:49.900274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:49.900274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:49.900475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:49.900759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] PATH_INFO: `/v3/projects/e66974f031994024a74dff17a449de5b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:49.901997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66974f031994024a74dff17a449de5b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:49.904043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:49.978643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212e555b56974e1bac17de03406bcf6c', 'request_id': 'req-a42eb3a9-d01d-41ba-af1a-222201a56228'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:49.986875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212e555b56974e1bac17de03406bcf6c', 'request_id': 'req-a42eb3a9-d01d-41ba-af1a-222201a56228'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.054715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66974f031994024a74dff17a449de5b', 'request_id': 'req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.066748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212e555b56974e1bac17de03406bcf6c', 'request_id': 'req-a42eb3a9-d01d-41ba-af1a-222201a56228'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.082068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] The token cache is being invalidate because project 212e555b56974e1bac17de03406bcf6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:50.082823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.083701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.086274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 944/1861] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:49 2022] DELETE /identity/v3/projects/212e555b56974e1bac17de03406bcf6c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:50.087673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.090069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.171262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb3a9-d01d-41ba-af1a-222201a56228 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.180972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66974f031994024a74dff17a449de5b', 'request_id': 'req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.190307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66974f031994024a74dff17a449de5b', 'request_id': 'req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.207665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] The token cache is being invalidate because project e66974f031994024a74dff17a449de5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:50.208048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.208896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-53ba9774-7c9e-4dc6-bca5-88bc99bb3ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:50.210811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 918/1862] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:49 2022] DELETE /identity/v3/projects/e66974f031994024a74dff17a449de5b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:50.211882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e82a5-e474-4c3f-a2a4-b4c60ec50ac9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.212172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e82a5-e474-4c3f-a2a4-b4c60ec50ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.212598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e82a5-e474-4c3f-a2a4-b4c60ec50ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.230691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-283e82a5-e474-4c3f-a2a4-b4c60ec50ac9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:50.231898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-283e82a5-e474-4c3f-a2a4-b4c60ec50ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.235350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ca5e0-10c9-46e2-8cea-4e7b5d6d2952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.236330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ca5e0-10c9-46e2-8cea-4e7b5d6d2952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.236646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ca5e0-10c9-46e2-8cea-4e7b5d6d2952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.244040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ca5e0-10c9-46e2-8cea-4e7b5d6d2952 admin admin] PATH_INFO: `/v3/projects/32c1183ac6904a0db4cb631c1a27b7dc/users/a98556109cf34a7fa1c4bef5d608fafc/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.244040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ca5e0-10c9-46e2-8cea-4e7b5d6d2952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c1183ac6904a0db4cb631c1a27b7dc, user_id=a98556109cf34a7fa1c4bef5d608fafc, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.244326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 919/1863] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:50.244326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9777f1-e0bf-4a92-83df-0294e4cb47b1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.244509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ca5e0-10c9-46e2-8cea-4e7b5d6d2952 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.244590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9777f1-e0bf-4a92-83df-0294e4cb47b1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.244590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9777f1-e0bf-4a92-83df-0294e4cb47b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.255290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 945/1864] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:50 2022] PUT /identity/v3/projects/32c1183ac6904a0db4cb631c1a27b7dc/users/a98556109cf34a7fa1c4bef5d608fafc/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:50.256869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb12a5-2e12-461f-97ea-014c56c6c899 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.257309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb12a5-2e12-461f-97ea-014c56c6c899 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.257696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb12a5-2e12-461f-97ea-014c56c6c899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.263413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3e9777f1-e0bf-4a92-83df-0294e4cb47b1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:50.264807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3e9777f1-e0bf-4a92-83df-0294e4cb47b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.301841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c5eb12a5-2e12-461f-97ea-014c56c6c899 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:50.301841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c5eb12a5-2e12-461f-97ea-014c56c6c899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.338574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 920/1865] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:50.338574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.338574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.338574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.351594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 946/1866] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:50.351594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.351594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.351594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.368533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] MFA Rules not processed for user `a98556109cf34a7fa1c4bef5d608fafc`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:50.369781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.370638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:50.372918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.385689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 947/1867] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:50.385689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.395172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6de4-4939-4160-8e9b-ea4a54215ea6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.400141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.400847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.401139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.401423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.409296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:50.409296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.409296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.421046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 948/1868] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:50.421046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.429011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772f6c-fbc7-4d52-9a79-f0f5b931977b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.434101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.434769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.435060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.435349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.438003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:50.438003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.438003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.454722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 949/1869] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:43:50.454722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.454949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 921/1870] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:50.458526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.467601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9820e51-a801-4f4f-993f-1c5dafac5054 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.472987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.473462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1710-b128-464e-b92c-b011d85542c2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.473729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.474128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.474413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.475113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.477050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.478218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.478874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.479168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.479608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.480307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.482227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.584021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 950/1871] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:50.584021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.584021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:50.624670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 922/1872] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:50.624670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.645769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2fd9d-a0fe-4a7f-8f14-4b98b1b661a1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.689470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cddf5-e178-468e-bfd8-94d319194101 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.695093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.695846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.698119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.698119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.698119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.699515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.703102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.703920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.704216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.704496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.774035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.775493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.788170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 923/1873] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:50.788170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.788461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 951/1874] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:50.789310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.802827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe389df-74b5-4f4d-bdb2-4b601d52e09f service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.806139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe437-fc04-4cdb-bd26-6b7e732b0a10 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.808068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.808783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.809083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.809367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.810115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.811824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.812036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.812714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.813005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.814607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.814733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.817538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.823691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 952/1875] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:50.827632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.840583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedefd8-c1fd-43e5-ad59-e43592085584 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.845747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.846419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.846711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.846996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.847888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.850145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.860944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 953/1876] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:50.865459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.875755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bad2e-376f-4d82-a32a-87d01863dde4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.881044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f9cea-de14-4bf2-8f9a-53a4e65f030c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.882094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f9cea-de14-4bf2-8f9a-53a4e65f030c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.882439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f9cea-de14-4bf2-8f9a-53a4e65f030c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.882803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f9cea-de14-4bf2-8f9a-53a4e65f030c admin admin] PATH_INFO: `/v3/projects/bac9bcfdd4ef4a6d87f593bde0c352c9/users/f862a8b7b4534d1ba7d65b4688c32b92/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.900103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f9cea-de14-4bf2-8f9a-53a4e65f030c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac9bcfdd4ef4a6d87f593bde0c352c9, user_id=f862a8b7b4534d1ba7d65b4688c32b92, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.903857 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f9cea-de14-4bf2-8f9a-53a4e65f030c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.930879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 924/1877] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:50 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:50.936562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.946984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6256c-b1b6-4467-a0f9-ca89d17fd07d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.951771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.952420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.952739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.953023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.953754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.955611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.966363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 925/1878] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:50.970152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:50.980308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637063f5-556a-4820-b104-42dfee88537e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:50.984985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:50.985548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:50.985835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:50.986115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:50.986821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:50.988401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:50.997780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 954/1879] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:50 2022] PUT /identity/v3/projects/bac9bcfdd4ef4a6d87f593bde0c352c9/users/f862a8b7b4534d1ba7d65b4688c32b92/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:51.002689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:51.003055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:51.003299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 926/1880] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:51.003415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:51.004162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:51.014339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ad04-8815-42cc-938b-465d839b42bb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:51.023500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:51.024860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:51.076133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:51.077051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:51.077339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:51.077617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] PATH_INFO: `/v3/projects/4113971a8ee241d0bd7b01a15d5e5192/users/c3004c0efa2b4ea99926b1b91abb68c6/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:51.092597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4113971a8ee241d0bd7b01a15d5e5192, user_id=c3004c0efa2b4ea99926b1b91abb68c6, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:51.095534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:51.095676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 955/1881] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:43:51.099582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:51.127378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:51.156854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59860-5a1e-4d6b-b3a6-02167f4c6bf5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:51.160559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:51.161478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:51.161641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:51.161891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:51.163792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:51.165790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 927/1882] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:51 2022] PUT /identity/v3/projects/4113971a8ee241d0bd7b01a15d5e5192/users/c3004c0efa2b4ea99926b1b91abb68c6/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:51.169176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:51.170345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:51.248017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:51.249724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0eb415-e264-4345-beee-93dbd0410b6f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:51.254003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1351793-b0b9-47d8-8f87-56d013999d2d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:51.258895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:51.259666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:51.260032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:51.261488 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 956/1883] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:51.262842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:51.263525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1f2e5-3233-4205-97b9-24b66192bdf7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:51.263971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1f2e5-3233-4205-97b9-24b66192bdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:51.264336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1f2e5-3233-4205-97b9-24b66192bdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:51.312244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-81d1f2e5-3233-4205-97b9-24b66192bdf7 None None] MFA Rules not processed for user `c3004c0efa2b4ea99926b1b91abb68c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:51.313837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81d1f2e5-3233-4205-97b9-24b66192bdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:51.345464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:51.347064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:51.364866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 928/1884] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:51.408063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 957/1885] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:51.420238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:51.421788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:51.480931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a4154-359c-4d65-8503-b09aea34ba76 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:51.484505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:51.485246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:51.485445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:51.485737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:51.494995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:51.496447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:51.506757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 929/1886] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:43:52.462867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:53.684109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:53.899946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:53.900466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:53.900569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:53.953788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] MFA Rules not processed for user `f862a8b7b4534d1ba7d65b4688c32b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:53.955841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:54.029204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:54.101254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 958/1887] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:54.203278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:54.213512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219be8c-8a8e-473e-949e-69df493ba750 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:54.217745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330dd0-a24b-4ab9-8064-d51a2438751a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:54.218649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330dd0-a24b-4ab9-8064-d51a2438751a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:54.219007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330dd0-a24b-4ab9-8064-d51a2438751a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:54.219290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330dd0-a24b-4ab9-8064-d51a2438751a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:54.243509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330dd0-a24b-4ab9-8064-d51a2438751a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:54.245062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330dd0-a24b-4ab9-8064-d51a2438751a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:54.281510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 930/1888] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:55.899315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:55.900890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:55.916622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa532302-2f25-4dbc-96a0-f14ed61a07f2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:55.921247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:55.922217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:55.922509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:55.930352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] PATH_INFO: `/v3/users/79cdbc9918c640b08d8c7a1ea5f688d4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:55.930352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cdbc9918c640b08d8c7a1ea5f688d4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:55.930352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:56.139917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cdbc9918c640b08d8c7a1ea5f688d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:56.149584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cdbc9918c640b08d8c7a1ea5f688d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:56.232984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cdbc9918c640b08d8c7a1ea5f688d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:56.233749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cdbc9918c640b08d8c7a1ea5f688d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:56.235525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cdbc9918c640b08d8c7a1ea5f688d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:56.263780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 959/1889] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:55 2022] DELETE /identity/v3/users/79cdbc9918c640b08d8c7a1ea5f688d4 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:56.270478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:56.270478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:56.270478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:56.286924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:56.288142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:56.355984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 931/1890] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:56.366704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:56.367741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:56.425842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fb82c-199d-4610-a396-cd93b64c545c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:56.429665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:56.430371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:56.430661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:56.430939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:56.440250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:56.442333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:56.452287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 960/1891] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:56.936412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:56.937959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:56.950274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab61e9-95d6-4af1-9d4c-74fd3dfdb21e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:56.964713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:56.965629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:56.966007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:56.966179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] PATH_INFO: `/v3/projects/712b0a9750b5472e8884f08800d662eb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:56.967476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=712b0a9750b5472e8884f08800d662eb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:56.969431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:57.123574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712b0a9750b5472e8884f08800d662eb', 'request_id': 'req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.226811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712b0a9750b5472e8884f08800d662eb', 'request_id': 'req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.233807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712b0a9750b5472e8884f08800d662eb', 'request_id': 'req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.245185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] The token cache is being invalidate because project 712b0a9750b5472e8884f08800d662eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:57.245486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.246153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.247894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 932/1892] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:56 2022] DELETE /identity/v3/projects/712b0a9750b5472e8884f08800d662eb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:57.251780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:57.266498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b681e-4532-4ac5-84d6-0ac56c7eef89 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:57.323899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:57.324883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:57.325179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:57.325443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] PATH_INFO: `/v3/users/a98556109cf34a7fa1c4bef5d608fafc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:57.326624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98556109cf34a7fa1c4bef5d608fafc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:57.328629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:57.618797 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98556109cf34a7fa1c4bef5d608fafc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.626534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98556109cf34a7fa1c4bef5d608fafc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.763398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98556109cf34a7fa1c4bef5d608fafc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.764085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98556109cf34a7fa1c4bef5d608fafc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.764830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f298f78b-bdbe-4252-b65a-e9f24edddfae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98556109cf34a7fa1c4bef5d608fafc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:57.783228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 961/1893] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:43:57 2022] DELETE /identity/v3/users/a98556109cf34a7fa1c4bef5d608fafc => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:58.313750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:58.324413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:58.335078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4f11b-5bb6-48a3-af5e-75bfb48ff95d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:58.402681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:58.403923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:58.404390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:58.404948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] PATH_INFO: `/v3/projects/32c1183ac6904a0db4cb631c1a27b7dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:58.406854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c1183ac6904a0db4cb631c1a27b7dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:58.409272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:58.483851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bc7c-a86d-40d5-a4d6-4df9dcf68dfe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:58.485117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bc7c-a86d-40d5-a4d6-4df9dcf68dfe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:58.485518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bc7c-a86d-40d5-a4d6-4df9dcf68dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:58.504913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8f32bc7c-a86d-40d5-a4d6-4df9dcf68dfe None None] MFA Rules not processed for user `f862a8b7b4534d1ba7d65b4688c32b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:58.506294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8f32bc7c-a86d-40d5-a4d6-4df9dcf68dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:58.550290 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c1183ac6904a0db4cb631c1a27b7dc', 'request_id': 'req-bb2e219b-de58-4412-a103-fd2bcbd092bf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:58.597115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 962/1894] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:58.643560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c1183ac6904a0db4cb631c1a27b7dc', 'request_id': 'req-bb2e219b-de58-4412-a103-fd2bcbd092bf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:58.650748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c1183ac6904a0db4cb631c1a27b7dc', 'request_id': 'req-bb2e219b-de58-4412-a103-fd2bcbd092bf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:58.662863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] The token cache is being invalidate because project 32c1183ac6904a0db4cb631c1a27b7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:43:58.663159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:58.663828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bb2e219b-de58-4412-a103-fd2bcbd092bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:43:58.665478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 933/1895] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:43:58 2022] DELETE /identity/v3/projects/32c1183ac6904a0db4cb631c1a27b7dc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:58.670693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:58.671039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:58.671326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:58.693001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:58.696987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:58.756951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:58.795392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 963/1896] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:58.803190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:58.803683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:58.803997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:58.825129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:58.826357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:58.835933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 934/1897] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:58.840338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:58.848859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:58.852725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea75526-8d7b-4d3b-9795-626eaf7b3cd8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:58.857515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:58.858254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:58.858614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:58.858985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:58.859792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:43:58.860176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:58.861904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:58.878073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 964/1898] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:43:58.879389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:58.890187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406d5f-b660-4724-8d57-df5195e88335 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:58.896147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:58.899627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:58.899627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:58.899627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:58.899627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:58.900191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:59.018377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 935/1899] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:58 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:59.023221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:59.035029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed6ec-bac9-48a4-8132-aaa903187465 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:59.104001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:59.104716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:59.105014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:59.105328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:59.106037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:59.107998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:59.265151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 965/1900] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:59 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:59.266233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:59.284359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b55d-46ef-4dde-b59c-5cb546a110bb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:59.290164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:59.291131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:59.291423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:59.291762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:59.292778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:59.294176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:59.306780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 936/1901] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:43:59.314845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:59.325804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702d52-8ae5-44b2-a634-0b4460719633 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:59.330650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301d2d4-9428-4817-b7ba-99415c8f2ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:59.331317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d2d4-9428-4817-b7ba-99415c8f2ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:59.331601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d2d4-9428-4817-b7ba-99415c8f2ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:59.331879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d2d4-9428-4817-b7ba-99415c8f2ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:59.333496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301d2d4-9428-4817-b7ba-99415c8f2ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:59.335110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301d2d4-9428-4817-b7ba-99415c8f2ce5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:59.344936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 966/1902] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:43:59.349333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:59.359231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbed74-3f6f-4e07-82e5-4780e66b4fb9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:59.363991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:59.364960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:59.365248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:59.365529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] PATH_INFO: `/v3/projects/d8931f16714548ce8cc3e4a018c01145/users/9e667b071ff84c28b8a23f1a39c03e28/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:59.381043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8931f16714548ce8cc3e4a018c01145, user_id=9e667b071ff84c28b8a23f1a39c03e28, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:59.383723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:59.544626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 937/1903] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:43:59 2022] PUT /identity/v3/projects/d8931f16714548ce8cc3e4a018c01145/users/9e667b071ff84c28b8a23f1a39c03e28/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:43:59.556464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:59.557021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:59.557102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:59.585063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] MFA Rules not processed for user `9e667b071ff84c28b8a23f1a39c03e28`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:43:59.586379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:59.695906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 967/1904] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:43:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:43:59.700661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:59.810576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9879e48-439b-409a-9788-f0fbb5f98213 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:59.834529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:59.835238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:59.840396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:59.840396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:59.840396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:59.846640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:59.982556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 938/1905] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:43:59 2022] POST /identity/v3/projects => generated 373 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:43:59.986527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:00.000174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a542d5-a7fc-46b0-abd4-62776245d265 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:00.066940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:00.067636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.067927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.068207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.068944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:00.071279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:00.263512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 968/1906] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:43:59 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:00.267506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:00.279560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de83cd-ab2f-421d-9635-8cd47f9abeb1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:00.289591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:00.290883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.291253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.291893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.293227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:00.295489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:00.311696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 939/1907] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:00.312945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:00.324399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350caca-5d5b-4ec7-83f4-cffad1beff89 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:00.329128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:00.330305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.330595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.330875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] PATH_INFO: `/v3/projects/fe40bee6efaf4647b0b073cf4c08c29d/users/7b967c657342491aafc0da7d86ca5745/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.347388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe40bee6efaf4647b0b073cf4c08c29d, user_id=7b967c657342491aafc0da7d86ca5745, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:00.350893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:00.550679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 969/1908] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:00 2022] PUT /identity/v3/projects/fe40bee6efaf4647b0b073cf4c08c29d/users/7b967c657342491aafc0da7d86ca5745/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:00.554641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:00.564566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6746c6-1249-4ed6-a796-6fe90bf1d745 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:00.620445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484cc0-f22f-4918-ace1-ffe5b76b6c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:00.621145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484cc0-f22f-4918-ace1-ffe5b76b6c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.621436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484cc0-f22f-4918-ace1-ffe5b76b6c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.621718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484cc0-f22f-4918-ace1-ffe5b76b6c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.622454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca484cc0-f22f-4918-ace1-ffe5b76b6c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:00.624043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca484cc0-f22f-4918-ace1-ffe5b76b6c66 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:00.634757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 940/1909] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:00.638953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:00.650830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f9e07-8e64-4917-ad26-e3f9731e62e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:00.655631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:00.656525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.656934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.657123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] PATH_INFO: `/v3/projects/fe40bee6efaf4647b0b073cf4c08c29d/users/7b967c657342491aafc0da7d86ca5745/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.659119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe40bee6efaf4647b0b073cf4c08c29d, user_id=7b967c657342491aafc0da7d86ca5745, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:00.661831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:00.674116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 970/1910] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:00 2022] PUT /identity/v3/projects/fe40bee6efaf4647b0b073cf4c08c29d/users/7b967c657342491aafc0da7d86ca5745/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:00.684445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.684763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.685060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.714255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] MFA Rules not processed for user `7b967c657342491aafc0da7d86ca5745`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:00.715493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:00.801416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 941/1911] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:00.812118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:00.813866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:00.873070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baec9c-05f7-4cb2-9dcf-595a4574d8c7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:00.876996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:00.877724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:00.878019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:00.878302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:00.936043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:00.937532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:00.947518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 971/1912] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:02.624174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:02.906807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:05.566833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:05.630260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:06.518018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:06.643389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:07.334985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:07.446904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:08.432726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:08.563403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:08.569331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:08.571044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:08.584960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275abed-0b53-4125-bbb9-09c0377eda5e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:08.588750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b95be-a0f8-4fd4-8f38-85b4185a168b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:08.589398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b95be-a0f8-4fd4-8f38-85b4185a168b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:08.589683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b95be-a0f8-4fd4-8f38-85b4185a168b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:08.590016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b95be-a0f8-4fd4-8f38-85b4185a168b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:08.600151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b95be-a0f8-4fd4-8f38-85b4185a168b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:08.601644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b95be-a0f8-4fd4-8f38-85b4185a168b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:08.611431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 942/1913] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:09.512416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:10.421107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:10.655405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:12.084026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:12.712044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:12.883461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:14.344359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:15.363517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:16.078082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:24.134957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=56 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:26.517067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:30.001246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:30.002792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:30.067649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf5416-4ae9-4e5f-9fd8-6e2cb7da5598 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:30.073170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:30.074051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:30.078658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:30.078658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] PATH_INFO: `/v3/users/c3004c0efa2b4ea99926b1b91abb68c6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:30.078658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3004c0efa2b4ea99926b1b91abb68c6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:30.078658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:30.405626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3004c0efa2b4ea99926b1b91abb68c6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:30.412710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3004c0efa2b4ea99926b1b91abb68c6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:30.516820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3004c0efa2b4ea99926b1b91abb68c6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:30.517461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3004c0efa2b4ea99926b1b91abb68c6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:30.518122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3004c0efa2b4ea99926b1b91abb68c6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:30.536148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 972/1914] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:30 2022] DELETE /identity/v3/users/c3004c0efa2b4ea99926b1b91abb68c6 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:30.541148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:30.541663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:30.541663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:30.559200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:30.560388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:30.646759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 943/1915] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:30.658612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:30.660127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:30.723054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf569-9d87-41ab-aa22-511dd6f8ed50 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:30.730627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3c14d-108d-4f00-ad09-b0b8adf2989c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:30.731349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3c14d-108d-4f00-ad09-b0b8adf2989c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:30.731566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3c14d-108d-4f00-ad09-b0b8adf2989c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:30.732080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3c14d-108d-4f00-ad09-b0b8adf2989c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:30.742830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3c14d-108d-4f00-ad09-b0b8adf2989c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:30.744309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3c14d-108d-4f00-ad09-b0b8adf2989c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:30.758398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 973/1916] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:31.314235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:31.315681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:31.335033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648e664-e88e-42eb-a075-1e3ca0360e12 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:31.341803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:31.342903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:31.343282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:31.343630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] PATH_INFO: `/v3/projects/4113971a8ee241d0bd7b01a15d5e5192` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:31.345001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4113971a8ee241d0bd7b01a15d5e5192)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:31.347048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:31.492864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4113971a8ee241d0bd7b01a15d5e5192', 'request_id': 'req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:31.583089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4113971a8ee241d0bd7b01a15d5e5192', 'request_id': 'req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:31.589613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4113971a8ee241d0bd7b01a15d5e5192', 'request_id': 'req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:31.601150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] The token cache is being invalidate because project 4113971a8ee241d0bd7b01a15d5e5192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:31.601441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:31.602100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f83a7fd0-090a-4b9c-a9b4-f3d19d81721d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:31.604195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 944/1917] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:31 2022] DELETE /identity/v3/projects/4113971a8ee241d0bd7b01a15d5e5192 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:31.609128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:31.609590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:31.609699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:31.628313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:31.629565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:31.648461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:31.695068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 974/1918] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:31.706292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:31.706598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:31.706884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:31.727217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:31.728548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:31.736582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 945/1919] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:31.741053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:31.751792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff383-0ea0-4f8b-ad1c-6c46bf479705 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:31.756706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:31.757402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:31.757905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:31.758619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:31.760125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:44:31.760470 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:31.762136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:31.773657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 975/1920] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:44:31.778134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:31.787438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae4e9c-52c8-43bf-aa92-dfbdbf69c2d9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:31.792096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:31.792784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:31.793074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:31.793356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:31.794063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:31.795986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:31.941319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 946/1921] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:31 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:31.945445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:31.957003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1949e7b-0964-4b3b-9123-23fd51f5c05e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:32.015185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:32.015868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:32.016155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:32.016435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:32.017173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:32.019094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:32.132185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 976/1922] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:31 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:32.136189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:32.147107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6268c6-f64a-4c9c-913e-71ee037b99d5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:32.151875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:32.152549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:32.152862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:32.155702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:32.155702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:32.155702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:32.167120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 947/1923] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:32.172218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:32.182996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646aeb-e4fb-469d-bb9e-5edbe98d18af admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:32.187759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107a85c-520f-4753-952e-d8bf38d38d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:32.188440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107a85c-520f-4753-952e-d8bf38d38d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:32.188760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107a85c-520f-4753-952e-d8bf38d38d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:32.189050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107a85c-520f-4753-952e-d8bf38d38d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:32.189790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107a85c-520f-4753-952e-d8bf38d38d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:32.191464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107a85c-520f-4753-952e-d8bf38d38d1b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:32.202016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 977/1924] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:32.207399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:32.219081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e538720-6235-49ec-bdcd-46cb967b2e26 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:32.232900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:32.232900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:32.232900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:32.232900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] PATH_INFO: `/v3/projects/9e2f14d491364824b60854550fa7414c/users/4e7da7b71993419492b15b350253cf31/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:32.249043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2f14d491364824b60854550fa7414c, user_id=4e7da7b71993419492b15b350253cf31, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:32.252109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:32.360728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 948/1925] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:32 2022] PUT /identity/v3/projects/9e2f14d491364824b60854550fa7414c/users/4e7da7b71993419492b15b350253cf31/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:32.374520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:32.374805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:32.375093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:32.406749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] MFA Rules not processed for user `4e7da7b71993419492b15b350253cf31`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:32.407914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:32.483043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 978/1926] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:32.494228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:32.496100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:32.521462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:32.557212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caa463-1aca-46cf-af2e-ed8bf1e4d04d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:32.560951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10470f1e-7a3d-4516-a265-0d0c0038855c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:32.561612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10470f1e-7a3d-4516-a265-0d0c0038855c service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:32.561900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10470f1e-7a3d-4516-a265-0d0c0038855c service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:32.562183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10470f1e-7a3d-4516-a265-0d0c0038855c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:32.571029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10470f1e-7a3d-4516-a265-0d0c0038855c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:32.573088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10470f1e-7a3d-4516-a265-0d0c0038855c service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:32.583148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 949/1927] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:38.055244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:38.081397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:38.144124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7b94f-44e4-4839-8b85-c7e6f6dcb7b4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:38.154653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:38.155551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:38.155844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:38.156126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] PATH_INFO: `/v3/users/4e7da7b71993419492b15b350253cf31` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:38.157367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7da7b71993419492b15b350253cf31)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:38.159588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:38.368654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7da7b71993419492b15b350253cf31'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:38.376467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7da7b71993419492b15b350253cf31'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:38.497631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7da7b71993419492b15b350253cf31'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:38.498412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7da7b71993419492b15b350253cf31'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:38.499127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7da7b71993419492b15b350253cf31'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:38.519176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 979/1928] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:38 2022] DELETE /identity/v3/users/4e7da7b71993419492b15b350253cf31 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:38.523851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:38.524394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:38.524394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:38.544360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:38.546121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:38.622473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 950/1929] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:38.634291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:38.638021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:38.703794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f054430-6c23-4029-b1be-b5021e272c15 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:38.708382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee32d-7417-4687-bfd2-ba766e5cbc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:38.708382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee32d-7417-4687-bfd2-ba766e5cbc73 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:38.708591 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee32d-7417-4687-bfd2-ba766e5cbc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:38.708832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee32d-7417-4687-bfd2-ba766e5cbc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:38.718542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee32d-7417-4687-bfd2-ba766e5cbc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:38.719943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee32d-7417-4687-bfd2-ba766e5cbc73 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:38.730426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 980/1930] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:39.196352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:39.198403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:39.210910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa14f5-d959-43e9-9128-b2d2b960af03 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:39.215804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:39.216831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:39.217168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:39.217434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] PATH_INFO: `/v3/projects/9e2f14d491364824b60854550fa7414c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:39.221026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2f14d491364824b60854550fa7414c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:39.221026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:39.347576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2f14d491364824b60854550fa7414c', 'request_id': 'req-16a9f779-4d3e-482e-9d21-733aed8c86e5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:39.432993 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2f14d491364824b60854550fa7414c', 'request_id': 'req-16a9f779-4d3e-482e-9d21-733aed8c86e5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:39.449438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2f14d491364824b60854550fa7414c', 'request_id': 'req-16a9f779-4d3e-482e-9d21-733aed8c86e5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:39.465014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] The token cache is being invalidate because project 9e2f14d491364824b60854550fa7414c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:39.465578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:39.466334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-16a9f779-4d3e-482e-9d21-733aed8c86e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:39.468387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 951/1931] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:39 2022] DELETE /identity/v3/projects/9e2f14d491364824b60854550fa7414c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:39.473149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:39.473623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:39.473732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:39.494868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:39.496293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:39.539069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:39.567838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 981/1932] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:39.576154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:39.576664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:39.576776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:39.598172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:39.599544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:39.609958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 952/1933] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:39.614778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:39.626181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63100e-1a50-43a5-bfa6-df8ea893f752 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:39.631178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:39.631863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:39.632153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:39.632432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:39.633446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:44:39.634080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:39.636463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:39.649473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 982/1934] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:44:39.653990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:39.664215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010da75b-50c2-4007-885f-c36a926ed3a5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:39.671013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:39.671693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:39.671983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:39.672284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:39.679873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:39.679873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:39.806035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 953/1935] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:39 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:39.810268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:39.828317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09011607-da08-4df1-981e-87ffd21e3897 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:39.915645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:39.916508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:39.916967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:39.917259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:39.918075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:39.920302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:39.976496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:39.978599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:39.996878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556f4a0-ba35-49c1-a2cb-138537969a5c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.004598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.005502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.005795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.006146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] PATH_INFO: `/v3/users/9e667b071ff84c28b8a23f1a39c03e28` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.007538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e667b071ff84c28b8a23f1a39c03e28)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.010771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.024617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 983/1936] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:39 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:40.026444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:40.029155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.040487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8dfd9-0650-42fc-adfc-ff91c962d0d6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.047151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.047864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.048157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.048444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.049295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.051979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.066058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 984/1937] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:40.074494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.086671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1a4ad-7ab3-4654-be94-68cdf71b8d86 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.091966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.092777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.093074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.093359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.094142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.095822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.109696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 985/1938] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:40.114156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.126017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc20ad-fd8a-41af-b075-567b52660d32 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.131286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978ceb-1a88-4c4b-a540-b8aca72adfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.132135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978ceb-1a88-4c4b-a540-b8aca72adfc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.132422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978ceb-1a88-4c4b-a540-b8aca72adfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.133071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978ceb-1a88-4c4b-a540-b8aca72adfc2 admin admin] PATH_INFO: `/v3/projects/57dd46fcf04b4e8688c741d4d1c4d2dc/users/e11d5bb70a94451b83a1d4c11a7b2e2a/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.151285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978ceb-1a88-4c4b-a540-b8aca72adfc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57dd46fcf04b4e8688c741d4d1c4d2dc, user_id=e11d5bb70a94451b83a1d4c11a7b2e2a, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.154220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978ceb-1a88-4c4b-a540-b8aca72adfc2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.338078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 986/1939] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:40 2022] PUT /identity/v3/projects/57dd46fcf04b4e8688c741d4d1c4d2dc/users/e11d5bb70a94451b83a1d4c11a7b2e2a/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:40.348670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.349008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.349291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.372030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e667b071ff84c28b8a23f1a39c03e28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:40.381146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] MFA Rules not processed for user `e11d5bb70a94451b83a1d4c11a7b2e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:40.382384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:40.384954 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e667b071ff84c28b8a23f1a39c03e28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:40.393969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e667b071ff84c28b8a23f1a39c03e28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:40.446058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:40.451209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 987/1940] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:40.460410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.520382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e667b071ff84c28b8a23f1a39c03e28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:40.521676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-93911d2a-83fe-4b68-a4e7-bb9fe02ea2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e667b071ff84c28b8a23f1a39c03e28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:40.530153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 954/1941] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:39 2022] DELETE /identity/v3/users/9e667b071ff84c28b8a23f1a39c03e28 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:40.534062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.535413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.535788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.556721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:40.558002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01fe3d-b6ec-498c-9471-a522a3965606 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.558447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:40.636295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 955/1942] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:40.641960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.644508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:40.662618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.662618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.662618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.662618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.662960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.664895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.732324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e975-6093-4c68-9157-afdb2584ae2f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.736300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.737068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.737450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.737797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.747950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.749569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.760985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 956/1943] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:40.772726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 988/1944] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:40 2022] POST /identity/v3/projects => generated 377 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:40.774542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.777370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:40.807334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263b46-5c28-4961-b5aa-aef5f49f717d service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.842604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:40.890673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.891280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.891560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.891890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.892737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.895143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:40.961686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:40.963997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:40.976993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4084543-73e4-4954-849a-169dde8fc19f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:40.984122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:40.985217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:40.985594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:40.986000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] PATH_INFO: `/v3/users/8eb0ed48dd034be0ad63373e4a151cd9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:40.987259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb0ed48dd034be0ad63373e4a151cd9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:40.989266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.063037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 957/1945] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:40 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:41.064433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.074719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54347b6-ca68-4754-b6cc-2a21a02674a5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.079604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.080555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.080988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.081245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.081983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.083679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.096741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 958/1946] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:41.111933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.128672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca244da-18de-4000-8330-d901c87750f2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.133721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.134416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.134698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.136395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.136395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.138075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.149269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 959/1947] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:41.154032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.163915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6e0be-8167-4d4e-8495-74c555c21b19 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.169558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb999e-5454-4ea7-a20e-41c014ebdde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.170466 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb999e-5454-4ea7-a20e-41c014ebdde2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.170781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb999e-5454-4ea7-a20e-41c014ebdde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.171080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb999e-5454-4ea7-a20e-41c014ebdde2 admin admin] PATH_INFO: `/v3/projects/e968d39a02764edbb066b5a84038aeac/users/2ee184e231e145569dfc66331fe4ae33/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.185868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb999e-5454-4ea7-a20e-41c014ebdde2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e968d39a02764edbb066b5a84038aeac, user_id=2ee184e231e145569dfc66331fe4ae33, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.188556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb999e-5454-4ea7-a20e-41c014ebdde2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.241605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb0ed48dd034be0ad63373e4a151cd9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.254272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb0ed48dd034be0ad63373e4a151cd9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.329715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 960/1948] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:41 2022] PUT /identity/v3/projects/e968d39a02764edbb066b5a84038aeac/users/2ee184e231e145569dfc66331fe4ae33/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:41.341034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.341323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.341609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.351633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb0ed48dd034be0ad63373e4a151cd9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.352376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb0ed48dd034be0ad63373e4a151cd9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.353195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb0ed48dd034be0ad63373e4a151cd9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.381729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 989/1949] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:40 2022] DELETE /identity/v3/users/8eb0ed48dd034be0ad63373e4a151cd9 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:41.382630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.385720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:41.392526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:41.393767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] MFA Rules not processed for user `2ee184e231e145569dfc66331fe4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:41.395007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:41.472728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fd8a9-4baa-4980-87d6-b426b862b7ac admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.480362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.481657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.482115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.482564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] PATH_INFO: `/v3/projects/d8931f16714548ce8cc3e4a018c01145` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.486183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 961/1950] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:41.486322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8931f16714548ce8cc3e4a018c01145)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.489743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.489937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.491316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:41.512534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cc61a-1a88-4c77-afbc-ca84bb2c23f0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.518862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.520097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.520570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.521756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.522767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.525213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.665821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 962/1951] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:41 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:41.670552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.680585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8931f16714548ce8cc3e4a018c01145', 'request_id': 'req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.690356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8931f16714548ce8cc3e4a018c01145', 'request_id': 'req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.759047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b5043-2169-4b82-9cfd-57d5a7efa1d3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.767305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:41.769611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.770311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.770603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.770887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.771778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.774090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.802511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8931f16714548ce8cc3e4a018c01145', 'request_id': 'req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.816557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] The token cache is being invalidate because project d8931f16714548ce8cc3e4a018c01145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:41.816893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.817568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:41.819410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 990/1952] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:41 2022] DELETE /identity/v3/projects/d8931f16714548ce8cc3e4a018c01145 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:41.819935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.821212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:41.891664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe9f59-179a-4c0f-b8f8-e87b0110c91e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.896368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.897279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.897565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.897843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] PATH_INFO: `/v3/projects/f5f3aeaf69e147fc9c2e04723cea0fd2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.899033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f3aeaf69e147fc9c2e04723cea0fd2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.901260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:41.922147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 963/1953] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:41 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:41.923000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:41.924827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:41.938122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bda79-b1f5-422e-be63-e6844dd7e0dc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:41.942896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:41.943778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:41.944064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:41.944341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] PATH_INFO: `/v3/users/7b967c657342491aafc0da7d86ca5745` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:41.945609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b967c657342491aafc0da7d86ca5745)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:41.947530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.196291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b967c657342491aafc0da7d86ca5745'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.197639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f3aeaf69e147fc9c2e04723cea0fd2', 'request_id': 'req-6446a50c-d82d-4319-a1e9-fb3a2a57442d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.204673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f3aeaf69e147fc9c2e04723cea0fd2', 'request_id': 'req-6446a50c-d82d-4319-a1e9-fb3a2a57442d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.209117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b967c657342491aafc0da7d86ca5745'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.217124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b967c657342491aafc0da7d86ca5745'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.277769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f3aeaf69e147fc9c2e04723cea0fd2', 'request_id': 'req-6446a50c-d82d-4319-a1e9-fb3a2a57442d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.288651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] The token cache is being invalidate because project f5f3aeaf69e147fc9c2e04723cea0fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:42.289002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.289683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.292057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 991/1954] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:41 2022] DELETE /identity/v3/projects/f5f3aeaf69e147fc9c2e04723cea0fd2 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:42.296273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.297665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:42.319170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:42.366007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a50c-d82d-4319-a1e9-fb3a2a57442d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.372376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd781-fed9-420f-8524-565c7097cc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.372376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd781-fed9-420f-8524-565c7097cc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.372376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd781-fed9-420f-8524-565c7097cc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.372376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd781-fed9-420f-8524-565c7097cc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.372936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd781-fed9-420f-8524-565c7097cc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.374539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd781-fed9-420f-8524-565c7097cc00 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.384932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 992/1955] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:42.386118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.386418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.386701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.407209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:42.408553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:42.414275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b967c657342491aafc0da7d86ca5745'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.414974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-18912e89-c6aa-401c-ad0d-4889c0eae5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b967c657342491aafc0da7d86ca5745'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:42.418235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 993/1956] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:42.419032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.430844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 964/1957] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:41 2022] DELETE /identity/v3/users/7b967c657342491aafc0da7d86ca5745 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:42.432657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.432972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.433358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.441131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02f467-077b-40b7-b88c-a855f2d79b84 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.453699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:42.455200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:42.507218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.508399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.508717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.509000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] PATH_INFO: `/v3/projects/50879755bf22439da6a677551105480c/users/d0f4fefcb9ad4544a4cfe416521f8e37/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.528312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] RBAC: Authorizing `identity:create_grant(project_id=50879755bf22439da6a677551105480c, user_id=d0f4fefcb9ad4544a4cfe416521f8e37, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.530410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 965/1958] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:42.531443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.534644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.544511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e6a4b-ea83-4674-a6e0-23e448ced704 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.549160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.549818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.550102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.550379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.551106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:44:42.551416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.553050 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.565317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 966/1959] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:44:42.569540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.580164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f38ab-1ca4-4b3d-9381-b8a0287d9ac3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.585028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.585668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.585960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.586242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.587045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.589744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.697663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 994/1960] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:42 2022] PUT /identity/v3/projects/50879755bf22439da6a677551105480c/users/d0f4fefcb9ad4544a4cfe416521f8e37/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:42.704675 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.706585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:42.732667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 967/1961] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:42 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:42.742810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.764171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:42.790667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff0c62-1dbb-4107-a886-a6e86e06cdca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.818204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e8a6-6868-4b4a-a460-21e5a0da5ebe admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.825369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.826070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.826362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.826760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.827395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.829477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.863070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.864074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.864362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.864641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.865456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.867054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.877658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 995/1962] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:42.881749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:42.891201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088087f-3906-4178-a1f8-88db2b845e91 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:42.895746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9323b-0fc7-4a3a-a980-4681f15b25d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:42.896595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9323b-0fc7-4a3a-a980-4681f15b25d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.896925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9323b-0fc7-4a3a-a980-4681f15b25d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.897209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9323b-0fc7-4a3a-a980-4681f15b25d4 admin admin] PATH_INFO: `/v3/projects/50879755bf22439da6a677551105480c/users/d0f4fefcb9ad4544a4cfe416521f8e37/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.899168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9323b-0fc7-4a3a-a980-4681f15b25d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50879755bf22439da6a677551105480c, user_id=d0f4fefcb9ad4544a4cfe416521f8e37, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:42.902758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9323b-0fc7-4a3a-a980-4681f15b25d4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:42.913843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 996/1963] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:42 2022] PUT /identity/v3/projects/50879755bf22439da6a677551105480c/users/d0f4fefcb9ad4544a4cfe416521f8e37/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:42.924993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:42.925294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:42.925580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:42.952085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] MFA Rules not processed for user `d0f4fefcb9ad4544a4cfe416521f8e37`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:42.953350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.006367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 968/1964] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:42 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:43.010434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.020953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769da7-6ae3-435e-804f-cda25ee23cf6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.028394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 997/1965] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:43.044220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.044220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.101544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.102310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.102732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.103092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.106125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.107984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.110492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751cb4f-1fbc-4b3f-a701-8a108aaf5c8b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.114938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.115945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.116234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.116522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.120172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.126539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 969/1966] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:43.126539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.126539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.139018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ea90a-24d9-4659-a167-3bb13a810570 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.145179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.146065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.146349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.146628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] PATH_INFO: `/v3/projects/fe40bee6efaf4647b0b073cf4c08c29d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.147846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe40bee6efaf4647b0b073cf4c08c29d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.149863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.189521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.191049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.201569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 998/1967] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:43.202581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.215641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b2159-c0ec-4b7e-a6ac-3952d52ee313 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.224974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.225669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.225961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.226243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.226983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.228631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.247783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 999/1968] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:43.249660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.260957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc0ae8-ff68-451d-b2ec-135001536328 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.266398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d07635-f818-4594-8c23-d4ef1b9ee6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.267430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07635-f818-4594-8c23-d4ef1b9ee6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.267850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07635-f818-4594-8c23-d4ef1b9ee6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.268197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d07635-f818-4594-8c23-d4ef1b9ee6dc admin admin] PATH_INFO: `/v3/projects/73bae9dc8c5b44a79d46bf667f6031da/users/2d5c797091db4d808c3d876f6f64d081/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.294201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d07635-f818-4594-8c23-d4ef1b9ee6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bae9dc8c5b44a79d46bf667f6031da, user_id=2d5c797091db4d808c3d876f6f64d081, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.297157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d07635-f818-4594-8c23-d4ef1b9ee6dc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.479368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1000/1969] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:43 2022] PUT /identity/v3/projects/73bae9dc8c5b44a79d46bf667f6031da/users/2d5c797091db4d808c3d876f6f64d081/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:43.487800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe40bee6efaf4647b0b073cf4c08c29d', 'request_id': 'req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:43.489626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.489937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.490211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.517560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] MFA Rules not processed for user `2d5c797091db4d808c3d876f6f64d081`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:43.518877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.585436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1001/1970] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:43.585672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe40bee6efaf4647b0b073cf4c08c29d', 'request_id': 'req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:43.593368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe40bee6efaf4647b0b073cf4c08c29d', 'request_id': 'req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:43.597427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.598980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.609021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] The token cache is being invalidate because project fe40bee6efaf4647b0b073cf4c08c29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:43.609311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:43.610062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f719a56a-8d6c-4c4a-9ed4-dccb80f5613e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:43.615662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 970/1971] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:43 2022] DELETE /identity/v3/projects/fe40bee6efaf4647b0b073cf4c08c29d => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:43.617096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ec3a-13d6-4f22-b813-23d5e8a84631 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.617420 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ec3a-13d6-4f22-b813-23d5e8a84631 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.617704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ec3a-13d6-4f22-b813-23d5e8a84631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.636025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1250ec3a-13d6-4f22-b813-23d5e8a84631 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:43.637296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1250ec3a-13d6-4f22-b813-23d5e8a84631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.679814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ad830-6d87-418d-9ddf-c5341fb983c2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.686725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.687347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.687626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.687903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.689749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.699824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 971/1972] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:43.701365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.703634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.709734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.710017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.710957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.716858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1002/1973] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:43.723590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:43.732309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:43.732309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:43.741197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 972/1974] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:43.745859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.758008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd624-db65-4319-9a00-a63e73f0ee1d service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.763327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.764029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.764313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.764642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.765523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:44:43.765834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.767512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.781432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1003/1975] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:44:43.785827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.796602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefc09a-c104-4887-87f0-f28fff029f71 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.802628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.803428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.803761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.806814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.806814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.806814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:43.890627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 973/1976] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:43 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:43.894958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:43.906388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e699e-1669-4b11-8d05-a76a5a1df47c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:43.965949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:43.966638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:43.966937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:43.967220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:43.967945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:43.969937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.081974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1004/1977] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:43 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:44.087263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.101042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc484d-4804-426e-97d4-ff1a93c8d731 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.106509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.107173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.107456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.110355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.110355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.110355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.124086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 974/1978] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:44.125440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.135590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c00cd-7237-414f-a9e7-7aa3cef0c0bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.140480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35d0e5-de12-4862-a5fe-f3089aa5456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.141181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35d0e5-de12-4862-a5fe-f3089aa5456e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.141473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35d0e5-de12-4862-a5fe-f3089aa5456e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.141757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35d0e5-de12-4862-a5fe-f3089aa5456e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.142603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35d0e5-de12-4862-a5fe-f3089aa5456e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.144215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35d0e5-de12-4862-a5fe-f3089aa5456e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.154276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1005/1979] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:44.158774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.169607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617973-0893-4488-b27f-58496b4bac27 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.174252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.175144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.175434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.175716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] PATH_INFO: `/v3/projects/658d066406c0484aa0b2b58aa1ebe4a5/users/43786ebf2e35404e83b6a7a66afd8955/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.191780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=658d066406c0484aa0b2b58aa1ebe4a5, user_id=43786ebf2e35404e83b6a7a66afd8955, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.195904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.219815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:44.303438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 975/1980] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:44 2022] PUT /identity/v3/projects/658d066406c0484aa0b2b58aa1ebe4a5/users/43786ebf2e35404e83b6a7a66afd8955/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:44.315040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.315342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.315628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.343804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] MFA Rules not processed for user `43786ebf2e35404e83b6a7a66afd8955`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:44.345094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:44.415064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1006/1981] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:44.420060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.492774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c8f5-c381-4f48-84c3-f7698b28d18c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.497606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.498461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.498757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.499043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.499778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.501822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.622882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 976/1982] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:44 2022] POST /identity/v3/projects => generated 369 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:44.624575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.636736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32df2fa-b3d8-4b44-8f9c-b306ea928655 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.701317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.702084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.702600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.702677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.703574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.708292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.826795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1007/1983] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:44 2022] POST /identity/v3/users => generated 291 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:44.830973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.842044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ce4f2-f70a-45c7-a44c-815fc116a86f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.847247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.847990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.848280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.848564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.849418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.850970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:44.863864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 977/1984] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:44.870167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:44.880158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53034-c537-48e5-9ec1-bff998988b41 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:44.884714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:44.885669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:44.885983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:44.886245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] PATH_INFO: `/v3/projects/d1ca8549162d4450ba308a36cb30ab39/users/fa29b8865cee4f95b640a89740de29c4/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:44.901459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ca8549162d4450ba308a36cb30ab39, user_id=fa29b8865cee4f95b640a89740de29c4, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:44.904958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:45.008513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1008/1985] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:44 2022] PUT /identity/v3/projects/d1ca8549162d4450ba308a36cb30ab39/users/fa29b8865cee4f95b640a89740de29c4/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:45.013216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:45.023314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790782ec-5984-4e4e-a5c9-e815c312da60 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:45.093697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5024b6f-6e15-45cc-bcbf-aea688641140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:45.094397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5024b6f-6e15-45cc-bcbf-aea688641140 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:45.094683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5024b6f-6e15-45cc-bcbf-aea688641140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:45.094958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5024b6f-6e15-45cc-bcbf-aea688641140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:45.095680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5024b6f-6e15-45cc-bcbf-aea688641140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:45.097303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5024b6f-6e15-45cc-bcbf-aea688641140 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:45.108720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 978/1986] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:45.111838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:45.118923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:45.124526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb12b7-f67a-4ec0-8747-64385384602d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:45.129200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:45.130084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:45.130404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:45.130704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] PATH_INFO: `/v3/projects/d1ca8549162d4450ba308a36cb30ab39/users/fa29b8865cee4f95b640a89740de29c4/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:45.132646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ca8549162d4450ba308a36cb30ab39, user_id=fa29b8865cee4f95b640a89740de29c4, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:45.135312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:45.146412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1009/1987] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:45 2022] PUT /identity/v3/projects/d1ca8549162d4450ba308a36cb30ab39/users/fa29b8865cee4f95b640a89740de29c4/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:45.157545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:45.157843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:45.158134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:45.186339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] MFA Rules not processed for user `fa29b8865cee4f95b640a89740de29c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:45.187885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:45.260040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 979/1988] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:45.272265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:45.273932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:45.337694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f871d-63ae-43fd-9898-b3b72bb669bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:45.342273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:45.342870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:45.343159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:45.344033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:45.357854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:45.359306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:45.369383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1010/1989] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:45 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:45.393293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:45.393293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:45.403715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e894a3-c124-42f7-a141-36ac23436935 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:45.407436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:45.408079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:45.408362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:45.408639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:45.411446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:45.474342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:45.476373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:45.487384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 980/1990] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:45.501656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:45.511117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e291862-b40a-4092-a001-b21faf02eb55 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:45.515642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:45.516285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:45.516573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:45.520338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:45.577615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:45.578979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:45.588983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1011/1991] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:45.869223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:45.969157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:46.733331 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:47.539614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:47.548536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f68fd4-7c1e-45ca-a696-d9a27673d528 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:47.552184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:47.552941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:47.553231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:47.553512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:47.555272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:47.608249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:47.629609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:47.631079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa512ccd-2361-4236-8f88-64996277d218 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:47.641119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 981/1992] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:50.939855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:50.979844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:51.003397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e1a49-0383-42c7-96bd-9d6e56550252 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:51.007915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:51.008822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:51.009111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:51.009390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] PATH_INFO: `/v3/users/43786ebf2e35404e83b6a7a66afd8955` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:51.010611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43786ebf2e35404e83b6a7a66afd8955)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:51.012518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:51.340197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43786ebf2e35404e83b6a7a66afd8955'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:51.350298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43786ebf2e35404e83b6a7a66afd8955'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:51.482749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43786ebf2e35404e83b6a7a66afd8955'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:51.483643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43786ebf2e35404e83b6a7a66afd8955'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:51.484640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43786ebf2e35404e83b6a7a66afd8955'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:51.507829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1012/1993] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:50 2022] DELETE /identity/v3/users/43786ebf2e35404e83b6a7a66afd8955 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:51.512317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:51.512795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:51.512915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:51.532554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:51.534113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:51.613203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 982/1994] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:51.623386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:51.625487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:51.684521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeacbb-9956-4470-8ab4-8e9b189b01b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:51.688992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:51.689659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:51.689949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:51.690231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:51.699859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:51.702195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:51.712669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1013/1995] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:52.166910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:52.166910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:52.171575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ccb21-e4ef-4024-9a9f-6864dc412dba None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:52.176670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:52.177663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:52.178018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:52.178370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] PATH_INFO: `/v3/projects/658d066406c0484aa0b2b58aa1ebe4a5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:52.179657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] RBAC: Authorizing `identity:delete_project(project_id=658d066406c0484aa0b2b58aa1ebe4a5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:52.182047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:52.314852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658d066406c0484aa0b2b58aa1ebe4a5', 'request_id': 'req-a0f1ccc5-2201-429a-909e-46256127119d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.453697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658d066406c0484aa0b2b58aa1ebe4a5', 'request_id': 'req-a0f1ccc5-2201-429a-909e-46256127119d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.460066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658d066406c0484aa0b2b58aa1ebe4a5', 'request_id': 'req-a0f1ccc5-2201-429a-909e-46256127119d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.473913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] The token cache is being invalidate because project 658d066406c0484aa0b2b58aa1ebe4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:52.474206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.474897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.476444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 983/1996] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:52 2022] DELETE /identity/v3/projects/658d066406c0484aa0b2b58aa1ebe4a5 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:52.480509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:52.491177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61892f-e4a9-4c13-8513-128f39672fa9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:52.546863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:52.553173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:52.554073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:52.554390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:52.554896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] PATH_INFO: `/v3/users/fa29b8865cee4f95b640a89740de29c4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:52.557759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa29b8865cee4f95b640a89740de29c4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:52.561010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:52.888601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa29b8865cee4f95b640a89740de29c4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.895820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa29b8865cee4f95b640a89740de29c4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.989161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa29b8865cee4f95b640a89740de29c4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.990223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa29b8865cee4f95b640a89740de29c4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:52.990993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38fc13ea-7c68-49ee-99f3-1d6ebe412ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa29b8865cee4f95b640a89740de29c4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:53.023028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1014/1997] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:52 2022] DELETE /identity/v3/users/fa29b8865cee4f95b640a89740de29c4 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:53.530198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:53.540986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:53.546886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1ccc5-2201-429a-909e-46256127119d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:53.610977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:53.611967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:53.612628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:53.612987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] PATH_INFO: `/v3/projects/d1ca8549162d4450ba308a36cb30ab39` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:53.614618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ca8549162d4450ba308a36cb30ab39)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:53.616716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:53.628364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:53.814300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ca8549162d4450ba308a36cb30ab39', 'request_id': 'req-4d67f163-111c-4f2b-ac4f-271c413162d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:53.948426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ca8549162d4450ba308a36cb30ab39', 'request_id': 'req-4d67f163-111c-4f2b-ac4f-271c413162d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:53.955900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ca8549162d4450ba308a36cb30ab39', 'request_id': 'req-4d67f163-111c-4f2b-ac4f-271c413162d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:53.967122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] The token cache is being invalidate because project d1ca8549162d4450ba308a36cb30ab39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:53.967451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:53.968091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4d67f163-111c-4f2b-ac4f-271c413162d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:53.970014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 984/1998] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:53 2022] DELETE /identity/v3/projects/d1ca8549162d4450ba308a36cb30ab39 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:53.974919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:53.975458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:53.975561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:53.993784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:53.994998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.061973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1015/1999] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:54.070132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.070517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.070804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.108903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:54.110217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.135280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 985/2000] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:54.136862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.145077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c654f-76b3-43e0-81ad-f89a0cc5dee1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.146436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.146436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.152704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.152704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.152704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:44:54.152704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.152704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.163204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1016/2001] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:44:54.168940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.178167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a678c-af43-4620-8bf8-314eddffdfde None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.183270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.183941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.184229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.189029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.189029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.189029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.196539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.196539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.210231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5a2eb-8ec7-4940-93ac-8e46db405014 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.215487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.216611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.217103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.217397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] PATH_INFO: `/v3/users/2d5c797091db4d808c3d876f6f64d081` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.218755 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5c797091db4d808c3d876f6f64d081)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.220758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.304662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 986/2002] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/projects => generated 399 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:54.311334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.384884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b47906-e19f-4eef-b413-a9635bcc8f9e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.390580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.391247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.391532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.391808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.392931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.395276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.493749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5c797091db4d808c3d876f6f64d081'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:54.500903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5c797091db4d808c3d876f6f64d081'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:54.503452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:54.535830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 987/2003] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:54.539813 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.550099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751738b-2c8e-43e8-8c17-e14317cb2286 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.555026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.555685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.555974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.556257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.557025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.558667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.569840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 988/2004] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:54.573926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.584927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71c0b-d228-48ae-8b44-18bc5b12a00c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.590022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.590690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.590983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.591346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.592807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.594447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.610056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 989/2005] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:54.614004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.616782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5c797091db4d808c3d876f6f64d081'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:54.621163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5c797091db4d808c3d876f6f64d081'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:54.621163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8c3021c7-3175-4dad-8411-91156cec80e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5c797091db4d808c3d876f6f64d081'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:54.625091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57e4b1-020f-4520-af74-0d6e809c04b6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.631503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f7afc-98b6-41b0-abe7-4e3340153f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.632419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f7afc-98b6-41b0-abe7-4e3340153f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.632727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f7afc-98b6-41b0-abe7-4e3340153f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.633021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f7afc-98b6-41b0-abe7-4e3340153f8b admin admin] PATH_INFO: `/v3/projects/207ac8b049914e568a3bc2861a4ec043/users/902f627821be4b01bd25a2896da2a9b2/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.645984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1017/2006] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:54 2022] DELETE /identity/v3/users/2d5c797091db4d808c3d876f6f64d081 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:54.650320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.650687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.651055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.652874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f7afc-98b6-41b0-abe7-4e3340153f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=207ac8b049914e568a3bc2861a4ec043, user_id=902f627821be4b01bd25a2896da2a9b2, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.656366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f7afc-98b6-41b0-abe7-4e3340153f8b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.669352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:54.671051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.732445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1018/2007] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:54.745434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.746993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.760258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 990/2008] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:54 2022] PUT /identity/v3/projects/207ac8b049914e568a3bc2861a4ec043/users/902f627821be4b01bd25a2896da2a9b2/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:54.772233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.772604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.772980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.811271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:54.816096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] MFA Rules not processed for user `902f627821be4b01bd25a2896da2a9b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:54.816771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.847573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d653d-6e8a-45dd-b927-e6a7853f76ca None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.907181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 991/2009] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:54.911558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:54.915126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:54.917643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.918490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.918791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.919074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.976522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:54.991110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947abf8-4e09-45a4-9abb-045b65d8bd62 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:54.993168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:54.994709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:54.995772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:54.996461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:54.996980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:54.997272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:54.998028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.000063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.006348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1019/2010] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:55.094613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 992/2011] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:54 2022] POST /identity/v3/projects => generated 401 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:55.098639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.108409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37092276-2296-4930-ac19-b67b814ab2dc service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.165880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.166561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.166852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.167134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.167835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.169871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.318547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1020/2012] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:55 2022] POST /identity/v3/users => generated 307 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:55.319957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.331910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34954a-891c-4443-927b-9a54b1a56267 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.337421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.338184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.338539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.338824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.339626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.342023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.358001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 993/2013] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:55.359195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.369485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faf5ef-1597-4e99-abf2-799b654acf45 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.374604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.375902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.381231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.381231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] PATH_INFO: `/v3/projects/9041cc73c3d44380877e3ba84405ad24/users/9dc5de7c8daf4a109e31b0d0360ce61e/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.392656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9041cc73c3d44380877e3ba84405ad24, user_id=9dc5de7c8daf4a109e31b0d0360ce61e, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.395347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.473242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.475099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:55.488988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc41d8-3701-4867-a5b2-7f6222966e4a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.493812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.494764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.495124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.495416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] PATH_INFO: `/v3/projects/73bae9dc8c5b44a79d46bf667f6031da` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.496630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73bae9dc8c5b44a79d46bf667f6031da)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.499489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.550224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1021/2014] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:55 2022] PUT /identity/v3/projects/9041cc73c3d44380877e3ba84405ad24/users/9dc5de7c8daf4a109e31b0d0360ce61e/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:55.555740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.624148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915a0e5-8040-4d2c-a618-d482b2898442 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.628844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.629541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.629828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.630105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.630830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.632441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.642887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1022/2015] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:55.647101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.657585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc9d7-c664-4ae1-a2ec-10748016cbd4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.660462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73bae9dc8c5b44a79d46bf667f6031da', 'request_id': 'req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:55.698692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:55.714485 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0b0e-26f4-4242-9f2b-25ec4b959842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.715385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0b0e-26f4-4242-9f2b-25ec4b959842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.715675 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0b0e-26f4-4242-9f2b-25ec4b959842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.715959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0b0e-26f4-4242-9f2b-25ec4b959842 admin admin] PATH_INFO: `/v3/projects/9041cc73c3d44380877e3ba84405ad24/users/9dc5de7c8daf4a109e31b0d0360ce61e/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.718742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0b0e-26f4-4242-9f2b-25ec4b959842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9041cc73c3d44380877e3ba84405ad24, user_id=9dc5de7c8daf4a109e31b0d0360ce61e, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.721438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0b0e-26f4-4242-9f2b-25ec4b959842 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.732959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1023/2016] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:55 2022] PUT /identity/v3/projects/9041cc73c3d44380877e3ba84405ad24/users/9dc5de7c8daf4a109e31b0d0360ce61e/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:55.743746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.744045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.744333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.771674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] MFA Rules not processed for user `9dc5de7c8daf4a109e31b0d0360ce61e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:55.772964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:55.802774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73bae9dc8c5b44a79d46bf667f6031da', 'request_id': 'req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:55.810043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73bae9dc8c5b44a79d46bf667f6031da', 'request_id': 'req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:55.818836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:55.823172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] The token cache is being invalidate because project 73bae9dc8c5b44a79d46bf667f6031da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:55.823172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:55.824281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e997b499-f4c2-4d2c-8bfc-a4fcc9f23b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:55.825972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 994/2017] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:55 2022] DELETE /identity/v3/projects/73bae9dc8c5b44a79d46bf667f6031da => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:55.831264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb64a4f-58d9-4303-989d-ddc596072b0d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.831600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb64a4f-58d9-4303-989d-ddc596072b0d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.831911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb64a4f-58d9-4303-989d-ddc596072b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.851800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9bb64a4f-58d9-4303-989d-ddc596072b0d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:55.852966 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9bb64a4f-58d9-4303-989d-ddc596072b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:55.854094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1024/2018] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:55.865320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.866693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:55.915249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 995/2019] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:55.923127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.923422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.929908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.935243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cac428-a2eb-4ce9-8081-69c738bb75e5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.939205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.939864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.940221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.940566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.944025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:55.945509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:55.955011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.956759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:55.961182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 996/2020] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:55.962105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:55.969344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1025/2021] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:55.984115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094c338-1465-49e0-b6a6-f3847e75da57 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:55.990228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:55.991016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:55.991369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:55.991716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:55.992530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:44:55.992943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:55.994619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.007924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 997/2022] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:44:56.012711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.031624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc06b2a-800d-45a4-ad08-b2c5e3dd191b service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.035541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.035541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:56.037536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.038798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.038798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.038798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.039462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.041453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.049697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df72a-2f52-4b4d-af0a-fd708d7f0721 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.055142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.056143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.056536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.056949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] PATH_INFO: `/v3/users/e11d5bb70a94451b83a1d4c11a7b2e2a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.058296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11d5bb70a94451b83a1d4c11a7b2e2a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.060432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.176603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1026/2023] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:56 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:56.181044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.234313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:56.251025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6bad-7ae6-47c0-a847-1735e9a6c9ac admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.255880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.256577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.256904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.257191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.257899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.259835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.319002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11d5bb70a94451b83a1d4c11a7b2e2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:56.333506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11d5bb70a94451b83a1d4c11a7b2e2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:56.340431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11d5bb70a94451b83a1d4c11a7b2e2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:56.360900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1027/2024] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:56 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:56.360900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.412309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c21df8-75cf-444c-9563-ef3195ec5d30 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.416035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.416710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.417006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.417289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.419821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:56.445509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11d5bb70a94451b83a1d4c11a7b2e2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:56.445509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11d5bb70a94451b83a1d4c11a7b2e2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:56.447547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 998/2025] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:56 2022] DELETE /identity/v3/users/e11d5bb70a94451b83a1d4c11a7b2e2a => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:56.448343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.449773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:56.480160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.481669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eef03d-5748-43cd-908c-dcd87e665d33 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.491843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1028/2026] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:44:56.494534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.494953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.495277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.509770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:56.518579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:56.519847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:56.522986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446e9c-ca07-4842-84a4-804e7b77d996 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.527923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.529249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1029/2027] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:44:56.529586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.530042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.530531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.531364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.533425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.539992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.542028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:56.545336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 999/2028] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:56.549945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.560377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ca9e1-1202-4ee8-9180-221b189fd7ab admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.565570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.566400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.566787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.567199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.568044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.569771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.585915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1000/2029] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:56.587205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.598685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314638-f32e-4de0-abca-d62e2c8f69c3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.610519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.611714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.612038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.612649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] PATH_INFO: `/v3/projects/46fc613f3c044d6c846311f58e6cbaa9/users/b589f139a0a0409892488b9055b15dd6/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.617048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709b5aa-1187-463b-b1e9-ffe53cef2c33 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.620820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cfe12-8373-48d9-b1ce-bd2efe313b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.621475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cfe12-8373-48d9-b1ce-bd2efe313b07 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.621764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cfe12-8373-48d9-b1ce-bd2efe313b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.622121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cfe12-8373-48d9-b1ce-bd2efe313b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.629539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fc613f3c044d6c846311f58e6cbaa9, user_id=b589f139a0a0409892488b9055b15dd6, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.632198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.632878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cfe12-8373-48d9-b1ce-bd2efe313b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.634337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cfe12-8373-48d9-b1ce-bd2efe313b07 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:56.644398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1030/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:56.722081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1001/2031] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:56 2022] PUT /identity/v3/projects/46fc613f3c044d6c846311f58e6cbaa9/users/b589f139a0a0409892488b9055b15dd6/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:56.734592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.734943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.735232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.766273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] MFA Rules not processed for user `b589f139a0a0409892488b9055b15dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:56.767511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:56.783308 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:56.857354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1031/2032] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:56.860526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:56.933901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773c60e-0122-4d46-a915-6ea21c0f941a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:56.939319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:56.939980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:56.940269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:56.940553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:56.941279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:56.943190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.207348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1002/2033] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:44:56 2022] POST /identity/v3/projects => generated 379 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:44:57.208560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.221519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e8979-5092-4522-a9da-e9b588f9a4a4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.297551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.298340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.298644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.298928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.299754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.302078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.422050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1032/2034] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:44:57 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:44:57.429734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.440931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fa28-e890-487e-8d37-46541ba2a5b4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.445682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.446382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.446678 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.446952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.447696 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.449440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.462003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1003/2035] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:44:57.466396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.476808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51023ea4-859a-402e-839b-714e77e0a2ca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.481566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.483011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.483412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.483702 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] PATH_INFO: `/v3/projects/77757e94c1fb4113a33b7df721c40169/users/684075d36dbe4a0d9668988ee1dec66f/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.499016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=77757e94c1fb4113a33b7df721c40169, user_id=684075d36dbe4a0d9668988ee1dec66f, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.501783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.562054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.566612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:57.585956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd960b7d-8c3c-48bb-bd03-eaada6de0f01 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.591351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.592227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.592335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.592587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] PATH_INFO: `/v3/projects/57dd46fcf04b4e8688c741d4d1c4d2dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.593840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] RBAC: Authorizing `identity:delete_project(project_id=57dd46fcf04b4e8688c741d4d1c4d2dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.595839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.660095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1033/2036] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:57 2022] PUT /identity/v3/projects/77757e94c1fb4113a33b7df721c40169/users/684075d36dbe4a0d9668988ee1dec66f/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:57.664008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.727582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88724021-09af-4a1a-99ce-759e837030cf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.733318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.733995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.734288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.734573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.735314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.736985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.747436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1034/2037] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:44:57.751684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.762260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2590b0-d6e1-4737-a735-1b284692f50f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.772461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57dd46fcf04b4e8688c741d4d1c4d2dc', 'request_id': 'req-3ff77566-5abd-4f0f-be08-3b993ca2fdac'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:57.777803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:57.826387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8697-0469-474c-a9f2-3c5f315a2429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.827396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8697-0469-474c-a9f2-3c5f315a2429 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.827685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8697-0469-474c-a9f2-3c5f315a2429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.827965 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8697-0469-474c-a9f2-3c5f315a2429 admin admin] PATH_INFO: `/v3/projects/77757e94c1fb4113a33b7df721c40169/users/684075d36dbe4a0d9668988ee1dec66f/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.830292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8697-0469-474c-a9f2-3c5f315a2429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77757e94c1fb4113a33b7df721c40169, user_id=684075d36dbe4a0d9668988ee1dec66f, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.833060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8697-0469-474c-a9f2-3c5f315a2429 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:57.844473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1035/2038] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:44:57 2022] PUT /identity/v3/projects/77757e94c1fb4113a33b7df721c40169/users/684075d36dbe4a0d9668988ee1dec66f/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:57.854567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.854915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.855177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.868352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57dd46fcf04b4e8688c741d4d1c4d2dc', 'request_id': 'req-3ff77566-5abd-4f0f-be08-3b993ca2fdac'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:57.875189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57dd46fcf04b4e8688c741d4d1c4d2dc', 'request_id': 'req-3ff77566-5abd-4f0f-be08-3b993ca2fdac'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:57.891497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] MFA Rules not processed for user `684075d36dbe4a0d9668988ee1dec66f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:44:57.891497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:57.891727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] The token cache is being invalidate because project 57dd46fcf04b4e8688c741d4d1c4d2dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:57.891727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:57.891727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:57.891727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1004/2039] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:57 2022] DELETE /identity/v3/projects/57dd46fcf04b4e8688c741d4d1c4d2dc => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:57.894167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.895716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:57.911237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77566-5abd-4f0f-be08-3b993ca2fdac admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:57.921313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:57.972412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1036/2040] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:44:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:44:57.981578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:57.981826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:57.982638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:57.982969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] PATH_INFO: `/v3/users/2ee184e231e145569dfc66331fe4ae33` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:57.984465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee184e231e145569dfc66331fe4ae33)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:57.984871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:57.986344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:57.986450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:58.048347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab1faf-5219-4838-b3c7-bd90c7e61165 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:58.051982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:58.052651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:58.053006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:58.053264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:58.064332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:58.069145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:58.075884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1037/2041] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:58.226617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee184e231e145569dfc66331fe4ae33'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:58.241436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee184e231e145569dfc66331fe4ae33'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:58.249462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee184e231e145569dfc66331fe4ae33'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:58.364740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:58.373892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534247e3-06c8-464b-a38e-d2484580ac8e service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:58.374896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee184e231e145569dfc66331fe4ae33'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:58.375823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee184e231e145569dfc66331fe4ae33'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:58.377590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:58.378238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:58.378524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:58.378802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:58.380875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:58.384387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1005/2042] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:57 2022] DELETE /identity/v3/users/2ee184e231e145569dfc66331fe4ae33 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:44:58.429739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:58.463925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:58.465266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:58.475861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1038/2043] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:44:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:44:58.944953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:58.946443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:44:59.015124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e09fa-0bc1-46bb-9a7e-9aa9294d4fb7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:59.019711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:59.020599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:59.025803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:59.025803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] PATH_INFO: `/v3/projects/e968d39a02764edbb066b5a84038aeac` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:59.025803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e968d39a02764edbb066b5a84038aeac)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:59.025803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:59.143171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:44:59.155079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e968d39a02764edbb066b5a84038aeac', 'request_id': 'req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.224712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e968d39a02764edbb066b5a84038aeac', 'request_id': 'req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.231429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e968d39a02764edbb066b5a84038aeac', 'request_id': 'req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.243623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] The token cache is being invalidate because project e968d39a02764edbb066b5a84038aeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:44:59.244034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.244895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.246526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1006/2044] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:44:58 2022] DELETE /identity/v3/projects/e968d39a02764edbb066b5a84038aeac => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:44:59.251404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:44:59.267936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a45390-aa3b-4654-bc27-812e14056b89 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:44:59.342813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:44:59.342813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:44:59.342813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:44:59.342813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] PATH_INFO: `/v3/users/d0f4fefcb9ad4544a4cfe416521f8e37` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:44:59.343192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f4fefcb9ad4544a4cfe416521f8e37)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:44:59.343192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:44:59.554003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f4fefcb9ad4544a4cfe416521f8e37'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.561623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f4fefcb9ad4544a4cfe416521f8e37'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.665297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f4fefcb9ad4544a4cfe416521f8e37'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.665924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f4fefcb9ad4544a4cfe416521f8e37'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.666534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ec7054-5e1c-42ce-949d-bf6fac2d21aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f4fefcb9ad4544a4cfe416521f8e37'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:44:59.685639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1039/2045] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:44:59 2022] DELETE /identity/v3/users/d0f4fefcb9ad4544a4cfe416521f8e37 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:00.155814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:00.157371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:00.170957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a877a-d1cf-4e3a-8a6b-be0832f3477e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:00.238870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:00.240185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:00.240316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:00.240630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] PATH_INFO: `/v3/projects/50879755bf22439da6a677551105480c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:00.242220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50879755bf22439da6a677551105480c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:00.244223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:00.402753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50879755bf22439da6a677551105480c', 'request_id': 'req-9f8339d2-e71a-4f67-808b-de339708ee10'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:00.493346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50879755bf22439da6a677551105480c', 'request_id': 'req-9f8339d2-e71a-4f67-808b-de339708ee10'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:00.500509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50879755bf22439da6a677551105480c', 'request_id': 'req-9f8339d2-e71a-4f67-808b-de339708ee10'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:00.518060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] The token cache is being invalidate because project 50879755bf22439da6a677551105480c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:00.518060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:00.518060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9f8339d2-e71a-4f67-808b-de339708ee10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:00.520553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1007/2046] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:00 2022] DELETE /identity/v3/projects/50879755bf22439da6a677551105480c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:00.522352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:00.522709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:00.523254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:00.546356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:00.547812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:00.600209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:00.612482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1040/2047] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:00.621319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:00.621612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:00.621892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:00.641200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:00.642668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:00.650405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1008/2048] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:00.655010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:00.664818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344dd3e-8958-4d63-9476-3f63d3112e79 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:00.670377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:00.670961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:00.671369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:00.672022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:00.672477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:00.672854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:00.674446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:00.687043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1041/2049] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:00.691323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:00.701072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509e2cd-f2a2-469b-8275-ff690d0f4916 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:00.706392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:00.707130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:00.707491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:00.707844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:00.708709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:00.710784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:00.866500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1009/2050] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:00 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:00.870358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:00.881224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24183680-e1c0-4828-b238-8a017ecd3d2c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:00.939152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:00.939744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:00.940032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:00.940313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:00.941054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:00.942977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:01.059434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1042/2051] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:00 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:01.063908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:01.074291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22682ea6-e1cc-421a-861b-4cb1698feced admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:01.080430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:01.081135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:01.081426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:01.081707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:01.082474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:01.084440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:01.097163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1010/2052] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:01.101344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:01.115842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ab8-be91-41ce-85ab-dcb5ef7c1547 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:01.120769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d836a5-689c-4940-be9d-acb5de44ff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:01.121440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d836a5-689c-4940-be9d-acb5de44ff06 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:01.121892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d836a5-689c-4940-be9d-acb5de44ff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:01.122278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d836a5-689c-4940-be9d-acb5de44ff06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:01.123186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d836a5-689c-4940-be9d-acb5de44ff06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:01.125024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d836a5-689c-4940-be9d-acb5de44ff06 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:01.135954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1043/2053] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:01.141149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:01.154812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ce63-afd5-423e-b118-dd936aa815fc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:01.160237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:01.161168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:01.161450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:01.161727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] PATH_INFO: `/v3/projects/cd20523ecc3540afb3fa9662b805d8e8/users/0c769307ad5e4e5da6f811e81d194842/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:01.170984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:01.183858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd20523ecc3540afb3fa9662b805d8e8, user_id=0c769307ad5e4e5da6f811e81d194842, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:01.187138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:01.297725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1011/2054] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:01 2022] PUT /identity/v3/projects/cd20523ecc3540afb3fa9662b805d8e8/users/0c769307ad5e4e5da6f811e81d194842/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:01.308174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:01.308480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:01.308799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:01.337811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] MFA Rules not processed for user `0c769307ad5e4e5da6f811e81d194842`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:01.339033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:01.405129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1044/2055] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:01.416572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:01.418338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:01.499760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94205a0e-c6c1-4c8a-ad97-bb5dcb17f9f2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:01.503909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6806ef-43dd-4ce1-bbd7-a6ceeead8d1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:01.508823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6806ef-43dd-4ce1-bbd7-a6ceeead8d1a service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:01.508823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6806ef-43dd-4ce1-bbd7-a6ceeead8d1a service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:01.508823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6806ef-43dd-4ce1-bbd7-a6ceeead8d1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:01.514089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6806ef-43dd-4ce1-bbd7-a6ceeead8d1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:01.515506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6806ef-43dd-4ce1-bbd7-a6ceeead8d1a service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:01.524192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1012/2056] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:02.154782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:02.156192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:02.211683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:02.225975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce19e46-cdc4-431e-98f0-e60ec4fefb76 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:02.231095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:02.233984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:02.234284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:02.234569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] PATH_INFO: `/v3/users/902f627821be4b01bd25a2896da2a9b2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:02.235881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902f627821be4b01bd25a2896da2a9b2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:02.237944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:02.576517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902f627821be4b01bd25a2896da2a9b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:02.584341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902f627821be4b01bd25a2896da2a9b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:02.668212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902f627821be4b01bd25a2896da2a9b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:02.669194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902f627821be4b01bd25a2896da2a9b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:02.670116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902f627821be4b01bd25a2896da2a9b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:02.691103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1045/2057] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:02 2022] DELETE /identity/v3/users/902f627821be4b01bd25a2896da2a9b2 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:02.702996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:02.703513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:02.703603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:02.725067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:02.726587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:02.795892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1013/2058] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:02.804342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:02.805844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:02.867324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e7d5-b828-4dc5-baf2-8081ad943471 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:02.870934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:02.871588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:02.871878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:02.872427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:02.881718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:02.883160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:02.894404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1046/2059] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:03.424845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:03.426226 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:03.439000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde68be-9668-46e8-a52b-974b41e5f88b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:03.444343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:03.445357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:03.445651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:03.445934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] PATH_INFO: `/v3/projects/207ac8b049914e568a3bc2861a4ec043` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:03.447247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] RBAC: Authorizing `identity:delete_project(project_id=207ac8b049914e568a3bc2861a4ec043)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:03.449331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:03.578079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207ac8b049914e568a3bc2861a4ec043', 'request_id': 'req-41905b42-87eb-4197-b6f2-eee15f42691a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:03.676517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207ac8b049914e568a3bc2861a4ec043', 'request_id': 'req-41905b42-87eb-4197-b6f2-eee15f42691a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:03.682694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207ac8b049914e568a3bc2861a4ec043', 'request_id': 'req-41905b42-87eb-4197-b6f2-eee15f42691a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:03.683779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:03.695044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] The token cache is being invalidate because project 207ac8b049914e568a3bc2861a4ec043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:03.695335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:03.695974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:03.697573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1014/2060] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:03 2022] DELETE /identity/v3/projects/207ac8b049914e568a3bc2861a4ec043 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:03.701817 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:03.711974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fceb2-93c3-4ada-a909-1c800f88bef9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:03.775967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:03.776995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:03.777284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:03.777646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] PATH_INFO: `/v3/users/9dc5de7c8daf4a109e31b0d0360ce61e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:03.779608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc5de7c8daf4a109e31b0d0360ce61e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:03.781798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:03.799585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:04.054545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc5de7c8daf4a109e31b0d0360ce61e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:04.061348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc5de7c8daf4a109e31b0d0360ce61e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:04.142181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc5de7c8daf4a109e31b0d0360ce61e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:04.142844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc5de7c8daf4a109e31b0d0360ce61e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:04.143494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3d25efe2-f16d-4f4f-955c-0a512fde068b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc5de7c8daf4a109e31b0d0360ce61e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:04.163653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1047/2061] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:03 2022] DELETE /identity/v3/users/9dc5de7c8daf4a109e31b0d0360ce61e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:04.768746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:04.768746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:04.782782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905b42-87eb-4197-b6f2-eee15f42691a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:04.844718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:04.845852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:04.846314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:04.846797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] PATH_INFO: `/v3/projects/9041cc73c3d44380877e3ba84405ad24` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:04.848127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9041cc73c3d44380877e3ba84405ad24)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:04.850207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:04.973844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9041cc73c3d44380877e3ba84405ad24', 'request_id': 'req-442793a0-0ffc-4d35-802f-89c052fae243'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:05.057426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9041cc73c3d44380877e3ba84405ad24', 'request_id': 'req-442793a0-0ffc-4d35-802f-89c052fae243'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:05.063456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9041cc73c3d44380877e3ba84405ad24', 'request_id': 'req-442793a0-0ffc-4d35-802f-89c052fae243'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:05.075985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] The token cache is being invalidate because project 9041cc73c3d44380877e3ba84405ad24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:05.076272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:05.077047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-442793a0-0ffc-4d35-802f-89c052fae243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:05.078600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1015/2062] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:04 2022] DELETE /identity/v3/projects/9041cc73c3d44380877e3ba84405ad24 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:05.089263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.089739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.089845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.108427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:05.109735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:05.168263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1048/2063] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:05.176552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.176927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.177408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.196608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:05.197878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:05.207499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1016/2064] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:05.211796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.222241 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdebdeb-e285-4740-b7c5-2e0cdf96c923 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.226837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.227490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.227780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.228062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.228833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:05.229151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.230746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:05.242860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1049/2065] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:05.248582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.259913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55072952-ac03-4d7e-bc8c-1cc7a23940bf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.264979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.265644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.265929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.266209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.266922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.268853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:05.386603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1017/2066] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:05 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:05.390922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.401122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d29b98-ce6f-4dfe-91e6-d236019715e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.457273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.457920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.458206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.458481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.459163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.461118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:05.563348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1050/2067] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:05 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:05.567454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.583405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaca209-3919-42c5-ac0c-c8fadef72835 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.589072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.589769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.590059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.590340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.591795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.593560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:05.605045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1018/2068] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:05.609048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.619950 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f29d-a438-4ac7-8929-de6fc9012eed admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.625600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4dabd-5d69-4994-a59d-cbe2c1c03878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.626266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4dabd-5d69-4994-a59d-cbe2c1c03878 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.626556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4dabd-5d69-4994-a59d-cbe2c1c03878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.626836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4dabd-5d69-4994-a59d-cbe2c1c03878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.627601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4dabd-5d69-4994-a59d-cbe2c1c03878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.629335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4dabd-5d69-4994-a59d-cbe2c1c03878 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:05.631652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:05.647367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1051/2069] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:05.648638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.660870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64717215-b92a-48de-89e0-62abbc4e7b19 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.665850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.666902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.667246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.667588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] PATH_INFO: `/v3/projects/19c05341ff0844febf46b0d6a054ae7d/users/175357a9425d49b8971f94653b8119f6/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.683063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c05341ff0844febf46b0d6a054ae7d, user_id=175357a9425d49b8971f94653b8119f6, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.685904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:05.786465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1019/2070] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:05 2022] PUT /identity/v3/projects/19c05341ff0844febf46b0d6a054ae7d/users/175357a9425d49b8971f94653b8119f6/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:05.797057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.797358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.797644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.824633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] MFA Rules not processed for user `175357a9425d49b8971f94653b8119f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:05.826589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:05.893656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1052/2071] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:05.897783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:05.919357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:05.963977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8def7-4064-4de1-96d7-b6d10462510d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:05.968658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:05.969349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:05.969641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:05.969926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:05.970625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:05.972566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.050101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1020/2072] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:05 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:06.053939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:06.065386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ee5e2-b5ac-4d6c-8fb5-9c865e88478d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:06.136031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:06.136740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.137001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.137277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.138743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:06.140802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.230806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1053/2073] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:06 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:06.235080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:06.245548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6d36f-be4e-42f6-aac4-3697e1022c1f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:06.250904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:06.251569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.251893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.252180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.252978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:06.261390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.266304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1021/2074] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:06.271217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:06.288017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a2898-a93e-4e0f-b982-777bf965f2a2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:06.292906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:06.293862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.294156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.294594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] PATH_INFO: `/v3/projects/78083a0977ff4812bfee525623e730d7/users/e31e61f7ce354dc4ae228adf5bdbc794/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.310076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78083a0977ff4812bfee525623e730d7, user_id=e31e61f7ce354dc4ae228adf5bdbc794, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:06.312833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.479509 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1054/2075] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:06 2022] PUT /identity/v3/projects/78083a0977ff4812bfee525623e730d7/users/e31e61f7ce354dc4ae228adf5bdbc794/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:06.484354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:06.494790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca17e5-f886-4e1b-a863-8631ae938c80 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:06.550507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7675637-17d4-44e5-b7dd-2da77d4b16c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:06.551168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7675637-17d4-44e5-b7dd-2da77d4b16c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.551455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7675637-17d4-44e5-b7dd-2da77d4b16c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.551732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7675637-17d4-44e5-b7dd-2da77d4b16c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.552455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7675637-17d4-44e5-b7dd-2da77d4b16c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:06.554090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7675637-17d4-44e5-b7dd-2da77d4b16c2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.568580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1022/2076] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:45:06.570285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:06.580545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8df26-74a8-4af5-aa6e-37c7b7e35792 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:06.585177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:06.586057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.586350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.586630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] PATH_INFO: `/v3/projects/78083a0977ff4812bfee525623e730d7/users/e31e61f7ce354dc4ae228adf5bdbc794/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.588584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78083a0977ff4812bfee525623e730d7, user_id=e31e61f7ce354dc4ae228adf5bdbc794, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:06.591288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.601252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1055/2077] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:06 2022] PUT /identity/v3/projects/78083a0977ff4812bfee525623e730d7/users/e31e61f7ce354dc4ae228adf5bdbc794/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:06.613001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.646861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.646861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.646861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] MFA Rules not processed for user `e31e61f7ce354dc4ae228adf5bdbc794`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:06.646861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:06.725026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1023/2078] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:06.737362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:06.739078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:06.786223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:06.803655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a3d67-9537-4ac3-ad1e-f961aae01ae1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:06.808465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:06.809204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:06.809567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:06.809892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:06.822567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:06.824988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:06.838209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1056/2079] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:07.107652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:07.109225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:07.121583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733fd0-1211-41da-9424-1f599b7c120f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:07.125295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f5a1c-4925-4f15-9dd5-b23b09b94fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:07.125986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f5a1c-4925-4f15-9dd5-b23b09b94fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:07.126288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f5a1c-4925-4f15-9dd5-b23b09b94fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:07.126550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f5a1c-4925-4f15-9dd5-b23b09b94fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:07.139064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:07.185713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f5a1c-4925-4f15-9dd5-b23b09b94fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:07.187228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f5a1c-4925-4f15-9dd5-b23b09b94fe1 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:07.197131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1024/2080] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:12.509133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:12.510578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:12.579183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de404a56-5a43-42db-aa4c-16fb30f8db83 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:12.584216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:12.585135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:12.585429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:12.585710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] PATH_INFO: `/v3/users/b589f139a0a0409892488b9055b15dd6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:12.587025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b589f139a0a0409892488b9055b15dd6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:12.588974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:12.779604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b589f139a0a0409892488b9055b15dd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:12.786761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b589f139a0a0409892488b9055b15dd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:12.867767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b589f139a0a0409892488b9055b15dd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:12.868402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b589f139a0a0409892488b9055b15dd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:12.869044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b589f139a0a0409892488b9055b15dd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:12.887661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1057/2081] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:12 2022] DELETE /identity/v3/users/b589f139a0a0409892488b9055b15dd6 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:12.893723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:12.894434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:12.894434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:12.920408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:12.921843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:12.981724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1025/2082] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:12.992014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:12.993436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:13.053067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f594d1-4c8f-4ca5-bcae-f16cd82cdaf8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:13.057251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:13.057928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:13.058230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:13.058521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:13.069435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:13.070900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:13.083242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1058/2083] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:13.550647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:13.556312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:13.564574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb93a1-4cb7-482d-8420-98f49626f6a7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:13.569766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:13.570635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:13.570913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:13.571188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] PATH_INFO: `/v3/projects/46fc613f3c044d6c846311f58e6cbaa9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:13.572389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46fc613f3c044d6c846311f58e6cbaa9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:13.574649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:13.734216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46fc613f3c044d6c846311f58e6cbaa9', 'request_id': 'req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:13.873637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46fc613f3c044d6c846311f58e6cbaa9', 'request_id': 'req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:13.880384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46fc613f3c044d6c846311f58e6cbaa9', 'request_id': 'req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:13.891581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] The token cache is being invalidate because project 46fc613f3c044d6c846311f58e6cbaa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:13.891874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:13.896365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:13.896365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1026/2084] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:13 2022] DELETE /identity/v3/projects/46fc613f3c044d6c846311f58e6cbaa9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:13.898288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:13.904294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:13.907977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5a01f-1340-4702-8502-16950968c5eb service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:13.966041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:13.967000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:13.967361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:13.967649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] PATH_INFO: `/v3/users/684075d36dbe4a0d9668988ee1dec66f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:13.969079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=684075d36dbe4a0d9668988ee1dec66f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:13.971598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:14.174690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684075d36dbe4a0d9668988ee1dec66f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.184097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684075d36dbe4a0d9668988ee1dec66f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.267367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684075d36dbe4a0d9668988ee1dec66f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.268168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684075d36dbe4a0d9668988ee1dec66f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.268796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684075d36dbe4a0d9668988ee1dec66f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.307469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1059/2085] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:13 2022] DELETE /identity/v3/users/684075d36dbe4a0d9668988ee1dec66f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:14.500120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:14.501710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:14.514801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd7fb-e3d8-4140-92da-7d7111e031f1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:14.552316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:14.577534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:14.578449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:14.579322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:14.579609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] PATH_INFO: `/v3/users/175357a9425d49b8971f94653b8119f6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:14.581149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=175357a9425d49b8971f94653b8119f6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:14.583074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:14.819783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175357a9425d49b8971f94653b8119f6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.837864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175357a9425d49b8971f94653b8119f6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.841020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:14.842881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:14.848544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175357a9425d49b8971f94653b8119f6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.861336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86bede-6c21-4e2c-bcca-59b424d263aa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:14.866897 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:14.867929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:14.868901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:14.869309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] PATH_INFO: `/v3/projects/77757e94c1fb4113a33b7df721c40169` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:14.870946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=77757e94c1fb4113a33b7df721c40169)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:14.878899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:14.953860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175357a9425d49b8971f94653b8119f6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.954390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3821662a-bbcc-496f-9955-949fe2a500d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175357a9425d49b8971f94653b8119f6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:14.961228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1027/2086] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:14 2022] DELETE /identity/v3/users/175357a9425d49b8971f94653b8119f6 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:14.965827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:14.966122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:14.966423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:14.983144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:14.984370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.010661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77757e94c1fb4113a33b7df721c40169', 'request_id': 'req-16293a4d-b001-4f41-9735-58acf114d5bc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:15.020019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77757e94c1fb4113a33b7df721c40169', 'request_id': 'req-16293a4d-b001-4f41-9735-58acf114d5bc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:15.048860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1028/2087] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:15.058365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.059687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.115594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db007ec-b91f-4cfc-b5d8-2f58eb9aae99 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.124153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:15.130646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77757e94c1fb4113a33b7df721c40169', 'request_id': 'req-16293a4d-b001-4f41-9735-58acf114d5bc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:15.144266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] The token cache is being invalidate because project 77757e94c1fb4113a33b7df721c40169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:15.144928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:15.145853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-16293a4d-b001-4f41-9735-58acf114d5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:15.148143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1060/2088] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:14 2022] DELETE /identity/v3/projects/77757e94c1fb4113a33b7df721c40169 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:15.153288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb33ea-554e-4f90-9b6a-f3c1b96874ba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.153661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb33ea-554e-4f90-9b6a-f3c1b96874ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.153839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb33ea-554e-4f90-9b6a-f3c1b96874ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.166963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:15.172341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8aeb33ea-554e-4f90-9b6a-f3c1b96874ba None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:15.172851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.173508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.173884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8aeb33ea-554e-4f90-9b6a-f3c1b96874ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.173988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.174489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.176092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.235810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1061/2089] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:15.245244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.245542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.245828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.246386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.253200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.264542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1029/2090] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:45:15.274406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:15.275913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.285177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1062/2091] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:15.289514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.299932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27895ba-ca75-4f1b-b9e1-29d95468ca70 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.304597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.305286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.305570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.305845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.306613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:15.306922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.308528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.324366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1030/2092] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:15.325569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.337975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686528e0-c8b7-4c38-ae7e-f196b709697f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.345056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.345769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.346207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.346490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.347208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.349184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.458070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1063/2093] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:15 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:15.461838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.477058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece18ad7-4b44-4a7d-86e6-368d171f6c15 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.559505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.560197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.560504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.560820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.561529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.563774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.717778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1031/2094] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:15 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:15.721961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.732554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3612d-882f-4ebf-aac9-99c585427ed6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.737440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.738121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.738414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.738696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.741940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.741940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.752404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1064/2095] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:15.756517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.768110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56e959-4792-4d01-a24f-87524c93f53d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.774237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.774237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.774237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.774237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.774726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.776340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.786716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1032/2096] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:15.791016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.805047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbdd31-584b-47fe-858a-145429eb829a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.809463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:15.811531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.813786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5e528-0b9a-46ff-8c04-c60d91f37c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.814759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5e528-0b9a-46ff-8c04-c60d91f37c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.815098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5e528-0b9a-46ff-8c04-c60d91f37c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.815385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5e528-0b9a-46ff-8c04-c60d91f37c81 admin admin] PATH_INFO: `/v3/projects/cff319722e054099bab1240ad00ef2c0/users/5f436275d0454b22b7a4a974b0c0ba2c/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.829553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b7a7-ba5c-45dd-bd59-cba696c057ff admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:15.833380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5e528-0b9a-46ff-8c04-c60d91f37c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff319722e054099bab1240ad00ef2c0, user_id=5f436275d0454b22b7a4a974b0c0ba2c, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.835210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:15.839578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.839578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.840443 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5e528-0b9a-46ff-8c04-c60d91f37c81 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.840976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] PATH_INFO: `/v3/projects/19c05341ff0844febf46b0d6a054ae7d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.840976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c05341ff0844febf46b0d6a054ae7d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:15.840976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:15.864718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:15.933419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1065/2097] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:15 2022] PUT /identity/v3/projects/cff319722e054099bab1240ad00ef2c0/users/5f436275d0454b22b7a4a974b0c0ba2c/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:15.944277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:15.944575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:15.944892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:15.971961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] MFA Rules not processed for user `5f436275d0454b22b7a4a974b0c0ba2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:15.977123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:15.984307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c05341ff0844febf46b0d6a054ae7d', 'request_id': 'req-cc22150a-ac8f-4bc4-9d36-dc193c06d730'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.051489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1066/2098] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:16.055978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.061932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c05341ff0844febf46b0d6a054ae7d', 'request_id': 'req-cc22150a-ac8f-4bc4-9d36-dc193c06d730'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.069750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c05341ff0844febf46b0d6a054ae7d', 'request_id': 'req-cc22150a-ac8f-4bc4-9d36-dc193c06d730'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.087192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] The token cache is being invalidate because project 19c05341ff0844febf46b0d6a054ae7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:16.087501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.088167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.089843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1033/2099] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:15 2022] DELETE /identity/v3/projects/19c05341ff0844febf46b0d6a054ae7d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:16.092701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:16.101360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.101360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:16.121004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a63da2-34e0-4142-b43d-f14988db08b8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.126365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.127022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.127306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.127586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.128274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.130317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.159362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22150a-ac8f-4bc4-9d36-dc193c06d730 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.163907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.164801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.165091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.165368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] PATH_INFO: `/v3/users/e31e61f7ce354dc4ae228adf5bdbc794` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.166576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31e61f7ce354dc4ae228adf5bdbc794)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.168467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.235919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1067/2100] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:16 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:16.239952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.241586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:16.310850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363be0a1-1564-4b52-a384-c582ff3a114b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.315604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.316281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.316571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.316895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.317663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.319629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.370154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31e61f7ce354dc4ae228adf5bdbc794'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.382752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31e61f7ce354dc4ae228adf5bdbc794'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.389887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31e61f7ce354dc4ae228adf5bdbc794'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.475777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1068/2101] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:16 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:16.480287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.495008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a6e30-7309-442b-81b7-8cbb52651e37 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.500043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31e61f7ce354dc4ae228adf5bdbc794'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.500043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31e61f7ce354dc4ae228adf5bdbc794'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:16.501307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.501307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.501307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.501520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.502380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.504015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.504400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1034/2102] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:16 2022] DELETE /identity/v3/users/e31e61f7ce354dc4ae228adf5bdbc794 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:16.522779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1069/2103] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:16.523927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.525752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:16.561968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:16.593899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3949fbb-fb03-4828-a831-7609618dd85e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.599664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.600566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.600890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.601174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] PATH_INFO: `/v3/projects/d9f1741967534b04b3149e9120eb0a89/users/443f21e2c7524765820e0adbf3797c6c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.615672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f1741967534b04b3149e9120eb0a89, user_id=443f21e2c7524765820e0adbf3797c6c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.618367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.705821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1035/2104] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:16 2022] PUT /identity/v3/projects/d9f1741967534b04b3149e9120eb0a89/users/443f21e2c7524765820e0adbf3797c6c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:16.717200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.729016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f01e09-f28b-4076-9422-a04649675d47 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.799216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1b88f-e2ae-4c7a-9464-bf75e6b614c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.799809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1b88f-e2ae-4c7a-9464-bf75e6b614c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.800104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1b88f-e2ae-4c7a-9464-bf75e6b614c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.804389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1b88f-e2ae-4c7a-9464-bf75e6b614c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.804389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1b88f-e2ae-4c7a-9464-bf75e6b614c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.804389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1b88f-e2ae-4c7a-9464-bf75e6b614c9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.815356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1070/2105] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:16.818392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.829257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9330cc-3658-44f1-ac68-75d3afccf4e9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:16.834217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:16.835179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.835531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.836218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] PATH_INFO: `/v3/projects/d9f1741967534b04b3149e9120eb0a89/users/443f21e2c7524765820e0adbf3797c6c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.838317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f1741967534b04b3149e9120eb0a89, user_id=443f21e2c7524765820e0adbf3797c6c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:16.840664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:16.851704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1036/2106] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:16 2022] PUT /identity/v3/projects/d9f1741967534b04b3149e9120eb0a89/users/443f21e2c7524765820e0adbf3797c6c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:16.862791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:16.863091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:16.863382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:16.892810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] MFA Rules not processed for user `443f21e2c7524765820e0adbf3797c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:16.894048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:16.924882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:16.981284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1071/2107] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:16.996864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:16.998717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:17.059928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d237ed1-c392-4971-a1cf-a303e2cb534c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:17.064787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391d9df-2717-4034-898b-7690546d5b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:17.065526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391d9df-2717-4034-898b-7690546d5b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:17.066060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391d9df-2717-4034-898b-7690546d5b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:17.066452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391d9df-2717-4034-898b-7690546d5b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:17.077027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391d9df-2717-4034-898b-7690546d5b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:17.078751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391d9df-2717-4034-898b-7690546d5b87 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:17.089605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1037/2108] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:17.154366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:17.157393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:17.236294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c08e02-3cb1-43f8-99ae-20bbd39e1b65 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:17.241110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e76839-2464-446c-9649-917006073d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:17.241974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e76839-2464-446c-9649-917006073d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:17.242264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e76839-2464-446c-9649-917006073d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:17.242546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e76839-2464-446c-9649-917006073d39 admin admin] PATH_INFO: `/v3/projects/78083a0977ff4812bfee525623e730d7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:17.243736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e76839-2464-446c-9649-917006073d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78083a0977ff4812bfee525623e730d7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:17.245723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e76839-2464-446c-9649-917006073d39 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:17.484804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e76839-2464-446c-9649-917006073d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78083a0977ff4812bfee525623e730d7', 'request_id': 'req-50e76839-2464-446c-9649-917006073d39'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:17.492247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e76839-2464-446c-9649-917006073d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78083a0977ff4812bfee525623e730d7', 'request_id': 'req-50e76839-2464-446c-9649-917006073d39'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:17.642330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e76839-2464-446c-9649-917006073d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78083a0977ff4812bfee525623e730d7', 'request_id': 'req-50e76839-2464-446c-9649-917006073d39'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:17.653164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e76839-2464-446c-9649-917006073d39 admin admin] The token cache is being invalidate because project 78083a0977ff4812bfee525623e730d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:17.653455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e76839-2464-446c-9649-917006073d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:17.654201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e76839-2464-446c-9649-917006073d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:17.655751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1072/2109] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:17 2022] DELETE /identity/v3/projects/78083a0977ff4812bfee525623e730d7 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:17.661736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:17.662033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:17.662318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:17.686691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:17.688864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:17.777428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1038/2110] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:17.783643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:17.783931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:17.784214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:17.802678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:17.803938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:17.813898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1073/2111] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:17.820798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:17.831447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4765d-db4d-4588-be33-60956e6e89f6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:17.836433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:17.837231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:17.837519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:17.837803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:17.838560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:17.838870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:17.840455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:17.853254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1039/2112] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:45:17.857661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:17.868142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570327da-d006-4d16-aaeb-83ef7ad44d8c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:17.872856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:17.873559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:17.877598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:17.877598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:17.877598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:17.877598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.050783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1074/2113] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:17 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:18.055356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.065974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a3d32-5fee-4057-a270-ff9029cac77c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.105978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:18.133388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.134063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.135111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.135645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.136728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.139105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.237111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1040/2114] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:18 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:18.241131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.256598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0f54f-54e2-4445-842d-0a56d62bef2d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.261279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.261947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.262237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.262518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.263554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.265529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.279211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1075/2115] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:45:18.290029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.299860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269176-734e-4267-b850-9ee0ab899e20 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.304888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.305843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.306136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.306417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.307157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.308875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.321449 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1041/2116] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:18.325647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.336276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3692da-1cb1-471a-943a-2b13e20b329f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.341277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e7bac-54ad-4c8f-b45d-8450d721e733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.342219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e7bac-54ad-4c8f-b45d-8450d721e733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.342513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e7bac-54ad-4c8f-b45d-8450d721e733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.342997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e7bac-54ad-4c8f-b45d-8450d721e733 admin admin] PATH_INFO: `/v3/projects/4e938f8032c5483db7fc9563fd860396/users/0d6cf90d357948b79cbab48dac554f34/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.346581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:18.357565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e7bac-54ad-4c8f-b45d-8450d721e733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e938f8032c5483db7fc9563fd860396, user_id=0d6cf90d357948b79cbab48dac554f34, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.360776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e7bac-54ad-4c8f-b45d-8450d721e733 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.435774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.437829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:18.505177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e98510-6a84-41bf-8f46-b7ef0d342184 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.509977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '443f21e2c7524765820e0adbf3797c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9f1741967534b04b3149e9120eb0a89', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.510653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.510948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.511235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.511979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:18.512297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.513956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.528224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1076/2117] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:18 2022] PUT /identity/v3/projects/4e938f8032c5483db7fc9563fd860396/users/0d6cf90d357948b79cbab48dac554f34/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:18.528536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1042/2118] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:45:18.529788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.530087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.530364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.536585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.548150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3931c-b4f9-4b64-9c4f-f65f49b82991 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.563518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] MFA Rules not processed for user `0d6cf90d357948b79cbab48dac554f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:18.564866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:18.612342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '443f21e2c7524765820e0adbf3797c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9f1741967534b04b3149e9120eb0a89', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.613034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.613317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.613593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.614289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.616371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.633326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1077/2119] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:18.645453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.646875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:18.704219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a567-bf52-4234-a782-b531101fc195 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.708028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.708695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.708997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.709290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.717936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.719534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.729842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1078/2120] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:45:18.746614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1043/2121] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:18 2022] POST /identity/v3/projects => generated 354 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:18.751567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:18.765812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262ecef-53ee-47a4-be47-14e72ea9e9d9 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:18.832665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '443f21e2c7524765820e0adbf3797c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9f1741967534b04b3149e9120eb0a89', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:18.833364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:18.833718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:18.834009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:18.834750 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:18.835065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:18.836699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:18.851600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1079/2122] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:19.451079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:19.482189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a16d-a2b0-4a12-b0d0-544442c5d515 tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:19.488887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '443f21e2c7524765820e0adbf3797c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9f1741967534b04b3149e9120eb0a89', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:19.490133 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:19.490418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:19.490695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] PATH_INFO: `/v3/projects/c73b3d102029444cbf5a70d3c5cdd5a1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:19.492334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c73b3d102029444cbf5a70d3c5cdd5a1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:19.494521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:19.524505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:19.652147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c73b3d102029444cbf5a70d3c5cdd5a1', 'request_id': 'req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:19.773641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c73b3d102029444cbf5a70d3c5cdd5a1', 'request_id': 'req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:19.780796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c73b3d102029444cbf5a70d3c5cdd5a1', 'request_id': 'req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:19.798223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] The token cache is being invalidate because project c73b3d102029444cbf5a70d3c5cdd5a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:19.798223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:19.798223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3be9100d-9da7-4ec5-8fdd-69f0eba5797c tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:19.798223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1044/2123] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:19 2022] DELETE /identity/v3/projects/c73b3d102029444cbf5a70d3c5cdd5a1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:20.597894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:20.599635 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:20.612935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:20.669911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cebfe4-5aed-4689-9b06-7aa2f7c2181b tempest-VolumeQuotaClassesTest-691588463 tempest-VolumeQuotaClassesTest-691588463-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:20.726542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:20.727415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:20.727698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:20.727974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] PATH_INFO: `/v3/users/5f436275d0454b22b7a4a974b0c0ba2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:20.729200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f436275d0454b22b7a4a974b0c0ba2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:20.731118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:20.929928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f436275d0454b22b7a4a974b0c0ba2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:20.936588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f436275d0454b22b7a4a974b0c0ba2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:20.998795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f436275d0454b22b7a4a974b0c0ba2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:20.999420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f436275d0454b22b7a4a974b0c0ba2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:21.000060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f436275d0454b22b7a4a974b0c0ba2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:21.020872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1080/2124] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:20 2022] DELETE /identity/v3/users/5f436275d0454b22b7a4a974b0c0ba2c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:21.025684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:21.026489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:21.026920 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:21.045323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:21.046537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:21.104689 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1045/2125] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:21.115763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:21.117223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:21.161493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:21.177811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f257d79-8b8b-4f20-88c0-273d100695dd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:21.181544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:21.182202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:21.182508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:21.182767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:21.192337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:21.193876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:21.204404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1081/2126] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:21.815317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:21.817060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:21.835328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e9ce2-2681-414c-a6b7-e08a21890e94 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:21.840901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:21.842031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:21.842447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:21.842797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] PATH_INFO: `/v3/projects/cff319722e054099bab1240ad00ef2c0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:21.844777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff319722e054099bab1240ad00ef2c0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:21.847206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:22.082077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff319722e054099bab1240ad00ef2c0', 'request_id': 'req-743c3057-9054-4c0d-bebf-5126c9a44292'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.207419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff319722e054099bab1240ad00ef2c0', 'request_id': 'req-743c3057-9054-4c0d-bebf-5126c9a44292'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.213711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff319722e054099bab1240ad00ef2c0', 'request_id': 'req-743c3057-9054-4c0d-bebf-5126c9a44292'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.224868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] The token cache is being invalidate because project cff319722e054099bab1240ad00ef2c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:22.225158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.225784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-743c3057-9054-4c0d-bebf-5126c9a44292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.227338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1046/2127] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:21 2022] DELETE /identity/v3/projects/cff319722e054099bab1240ad00ef2c0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:22.231309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:22.241597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5543a16-6390-48be-b8d2-4edf7df986bf service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:22.304330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:22.305249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:22.305537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:22.305814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] PATH_INFO: `/v3/users/443f21e2c7524765820e0adbf3797c6c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:22.306999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=443f21e2c7524765820e0adbf3797c6c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:22.309083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:22.570171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:22.570492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:22.570793 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:22.589207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] MFA Rules not processed for user `b86d6011a3f04ad0bf5b8be696c789f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:22.590404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:22.612556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443f21e2c7524765820e0adbf3797c6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.619856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443f21e2c7524765820e0adbf3797c6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.622220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:22.647433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1047/2128] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:22.692718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443f21e2c7524765820e0adbf3797c6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.693457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443f21e2c7524765820e0adbf3797c6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.694073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-75a3dc8f-7ca5-4cec-931d-bdffedfb8602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443f21e2c7524765820e0adbf3797c6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:22.713751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1082/2129] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:22 2022] DELETE /identity/v3/users/443f21e2c7524765820e0adbf3797c6c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:23.222125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:23.223591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:23.239314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08481021-9c4f-4bf5-b69f-7102984c3d35 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:23.313665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:23.320269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:23.320269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:23.320269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] PATH_INFO: `/v3/projects/d9f1741967534b04b3149e9120eb0a89` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:23.320269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f1741967534b04b3149e9120eb0a89)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:23.320269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:23.491713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f1741967534b04b3149e9120eb0a89', 'request_id': 'req-579af814-bd51-4fbf-9e9b-77ffaff523f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:23.736830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f1741967534b04b3149e9120eb0a89', 'request_id': 'req-579af814-bd51-4fbf-9e9b-77ffaff523f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:23.743301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f1741967534b04b3149e9120eb0a89', 'request_id': 'req-579af814-bd51-4fbf-9e9b-77ffaff523f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:23.778661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] The token cache is being invalidate because project d9f1741967534b04b3149e9120eb0a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:23.778963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:23.785188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-579af814-bd51-4fbf-9e9b-77ffaff523f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:23.785188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1048/2130] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:23 2022] DELETE /identity/v3/projects/d9f1741967534b04b3149e9120eb0a89 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:23.786318 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:23.786747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:23.786937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:23.808747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:23.809859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:23.883611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:23.892664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1083/2131] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:23.905413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:23.905853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:23.906752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:23.929285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:23.930413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:23.940749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1049/2132] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:23.944950 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:23.956278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b64ea-3253-473e-b40c-126374116f19 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:23.961396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:23.962110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:23.962404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:23.962689 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:23.963579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:23.963896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:23.965901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:23.978916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1084/2133] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:23.983212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:23.993106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783edbc0-3a0a-4fd9-80be-b8a85a0a330a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:23.997732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:23.998371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:23.998652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:23.998986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:23.999615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:24.002345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:24.123469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1050/2134] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:23 2022] POST /identity/v3/projects => generated 409 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:24.127726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:24.138031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede712-34e1-49d4-90f0-258f8bd4c42a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:24.211182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:24.212238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:24.212766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:24.213058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:24.213786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:24.215753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:24.308937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1085/2135] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:24 2022] POST /identity/v3/users => generated 305 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:24.312383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:24.323829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b2f4-0510-49f6-8f93-e4aa7b46dc3b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:24.328501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:24.329246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:24.329538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:24.330073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:24.330806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:24.332395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:24.345734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1051/2136] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:24.348533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:24.358718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ede349-dd80-4fb9-b651-dfe2c5187e99 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:24.363872 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdbb1f-afde-4e38-8ccd-5458c498f092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:24.364513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdbb1f-afde-4e38-8ccd-5458c498f092 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:24.364843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdbb1f-afde-4e38-8ccd-5458c498f092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:24.365126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdbb1f-afde-4e38-8ccd-5458c498f092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:24.365855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdbb1f-afde-4e38-8ccd-5458c498f092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:24.367535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdbb1f-afde-4e38-8ccd-5458c498f092 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:24.388119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1086/2137] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:24.392325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:24.409545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1ace7-51f9-4412-ad22-02dfe13476fa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:24.414758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:24.415778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:24.416142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:24.416438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] PATH_INFO: `/v3/projects/e78d5ab653cf4d1db63a7171a4e0b2b9/users/ec0c15eb638f479d8d40b8b8d5eb7ecb/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:24.432444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78d5ab653cf4d1db63a7171a4e0b2b9, user_id=ec0c15eb638f479d8d40b8b8d5eb7ecb, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:24.435128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:24.578042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1052/2138] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:24 2022] PUT /identity/v3/projects/e78d5ab653cf4d1db63a7171a4e0b2b9/users/ec0c15eb638f479d8d40b8b8d5eb7ecb/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:24.598188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:24.598419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:24.598722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:24.625691 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] MFA Rules not processed for user `ec0c15eb638f479d8d40b8b8d5eb7ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:24.627056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:24.692289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1087/2139] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:24.697158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:24.706942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:24.759205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b429045-8b27-488c-af65-481365c9780e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:24.764022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:24.764726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:24.765021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:24.765302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:24.766233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:24.768170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:24.934948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1053/2140] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:24 2022] POST /identity/v3/projects => generated 405 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:24.949971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:24.960996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d749cb-1e40-409f-bc94-5dd6c09db2b9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:25.017683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:25.018350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.018636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.018914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.019610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:25.021607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:25.116208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1088/2141] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:24 2022] POST /identity/v3/users => generated 309 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:25.121863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:25.132411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd602d87-28cc-46ff-8797-aa0f3ccc4504 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:25.137088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:25.137740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.138024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.138301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.139025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:25.141308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:25.152866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1054/2142] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:25.157849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:25.168038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759140f3-9ee8-4494-9c89-b8e73afd860d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:25.172953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:25.173875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.174158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.174434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] PATH_INFO: `/v3/projects/344fab8c44b6495792d747e2e65f61c3/users/790f49d59001495cbfa33783a4983ed1/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.188819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344fab8c44b6495792d747e2e65f61c3, user_id=790f49d59001495cbfa33783a4983ed1, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:25.191463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:25.290113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1089/2143] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:25 2022] PUT /identity/v3/projects/344fab8c44b6495792d747e2e65f61c3/users/790f49d59001495cbfa33783a4983ed1/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:25.294291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:25.314597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1bb4d-4176-43a6-b50c-aa83c1277bff admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:25.372002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb97d8-d814-49f1-b542-8541d4dec5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:25.372575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb97d8-d814-49f1-b542-8541d4dec5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.372895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb97d8-d814-49f1-b542-8541d4dec5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.373204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb97d8-d814-49f1-b542-8541d4dec5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.373937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb97d8-d814-49f1-b542-8541d4dec5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:25.375558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb97d8-d814-49f1-b542-8541d4dec5b7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:25.385140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1055/2144] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:25.389622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:25.400259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158852af-866d-4e91-9492-909e419b9f01 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:25.405216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:25.406089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.406899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.407176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] PATH_INFO: `/v3/projects/344fab8c44b6495792d747e2e65f61c3/users/790f49d59001495cbfa33783a4983ed1/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.409250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344fab8c44b6495792d747e2e65f61c3, user_id=790f49d59001495cbfa33783a4983ed1, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:25.411854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:25.422669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1090/2145] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:25 2022] PUT /identity/v3/projects/344fab8c44b6495792d747e2e65f61c3/users/790f49d59001495cbfa33783a4983ed1/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:25.433900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.434201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.455041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.460434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] MFA Rules not processed for user `790f49d59001495cbfa33783a4983ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:25.461684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:25.530795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1056/2146] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:25 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:25.542421 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:25.544123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:25.604603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301750e-4955-4af6-99b1-9ab9deed5ad9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:25.608243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:25.608925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:25.609217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:25.609580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:25.617780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:25.620100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:25.621572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:25.631727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1091/2147] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:25 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:26.145145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:26.154803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6ff8-e306-4662-b257-d86f0ca2017a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:26.157839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:26.160916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe8d2-6fac-40df-9c05-bf551fbe074b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:26.161593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe8d2-6fac-40df-9c05-bf551fbe074b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:26.161884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe8d2-6fac-40df-9c05-bf551fbe074b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:26.162160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe8d2-6fac-40df-9c05-bf551fbe074b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:26.221011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbe8d2-6fac-40df-9c05-bf551fbe074b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:26.222520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbe8d2-6fac-40df-9c05-bf551fbe074b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:26.231597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1057/2148] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:36.201927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:36.404285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=27 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:43.791865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:43.793464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:43.879795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf22c1-3fef-4970-8f9e-897eaf55cd80 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:43.884736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:43.885579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:43.885866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:43.886140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] PATH_INFO: `/v3/users/0d6cf90d357948b79cbab48dac554f34` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:43.887341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6cf90d357948b79cbab48dac554f34)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:43.889287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:44.173109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6cf90d357948b79cbab48dac554f34'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:44.180553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6cf90d357948b79cbab48dac554f34'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:44.293312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6cf90d357948b79cbab48dac554f34'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:44.293920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6cf90d357948b79cbab48dac554f34'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:44.294542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6cf90d357948b79cbab48dac554f34'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:44.314230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1092/2149] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:43 2022] DELETE /identity/v3/users/0d6cf90d357948b79cbab48dac554f34 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:44.318874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:44.319776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:44.320060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:44.338057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:44.339301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:44.401365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1058/2150] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:44.411144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:44.413418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:44.470678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7203-9afb-4965-8e87-a467ccbd1d7c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:44.474458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:44.475115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:44.475405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:44.475694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:44.488839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:44.490344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:44.501215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1093/2151] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:44.975581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:44.977383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:44.998913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5484a06-1ea7-4999-80f8-c0650803b4a0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:45.009672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:45.010690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.011116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.011416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] PATH_INFO: `/v3/projects/4e938f8032c5483db7fc9563fd860396` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.012639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e938f8032c5483db7fc9563fd860396)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:45.014777 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:45.229700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e938f8032c5483db7fc9563fd860396', 'request_id': 'req-ba442912-06ea-4367-9763-be1fd2dbc067'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.255218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:45.267992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:45.286354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9440e83-6242-4ed5-9107-5ced64d7d46b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:45.330729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:45.353262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e938f8032c5483db7fc9563fd860396', 'request_id': 'req-ba442912-06ea-4367-9763-be1fd2dbc067'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.361987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e938f8032c5483db7fc9563fd860396', 'request_id': 'req-ba442912-06ea-4367-9763-be1fd2dbc067'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.379195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] The token cache is being invalidate because project 4e938f8032c5483db7fc9563fd860396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:45.379855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.380520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba442912-06ea-4367-9763-be1fd2dbc067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.382625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1059/2152] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:44 2022] DELETE /identity/v3/projects/4e938f8032c5483db7fc9563fd860396 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:45.387274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770a25-b07a-45d9-b4c1-3d2a1ad3df7d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.387572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770a25-b07a-45d9-b4c1-3d2a1ad3df7d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.387881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770a25-b07a-45d9-b4c1-3d2a1ad3df7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.393391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:45.394492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.394973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.395328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] PATH_INFO: `/v3/users/ec0c15eb638f479d8d40b8b8d5eb7ecb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.397738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0c15eb638f479d8d40b8b8d5eb7ecb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:45.409634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6f770a25-b07a-45d9-b4c1-3d2a1ad3df7d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:45.410867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6f770a25-b07a-45d9-b4c1-3d2a1ad3df7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:45.415507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:45.420273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1060/2153] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:45.429096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.429325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.429562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.460066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:45.461841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:45.479683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1061/2154] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:45.481059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:45.491051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2505730-b417-4122-aa4c-26b1a246adeb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:45.495987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:45.496650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.497039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.497319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.498060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:45.498369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:45.500193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:45.516781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1062/2155] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:45.522582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:45.531377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48accddd-89db-441d-acdb-5813dcfa8477 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:45.539623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:45.541379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.541771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.542130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.543001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:45.545197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:45.674524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0c15eb638f479d8d40b8b8d5eb7ecb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.681292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0c15eb638f479d8d40b8b8d5eb7ecb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.701795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1063/2156] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:45 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:45.704423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:45.747051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:45.772468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52092-3e33-457e-87c5-994d3be4abcf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:45.777887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:45.778929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.779406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.779834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.779950 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0c15eb638f479d8d40b8b8d5eb7ecb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.781147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0c15eb638f479d8d40b8b8d5eb7ecb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.782654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:45.785124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:45.786319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d353f859-3fa2-4f05-8830-0775db1f1d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0c15eb638f479d8d40b8b8d5eb7ecb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:45.840086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1094/2157] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:45 2022] DELETE /identity/v3/users/ec0c15eb638f479d8d40b8b8d5eb7ecb => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:45.844598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.845041 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.845628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.870319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:45.870319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:45.900722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1064/2158] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:45 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:45.901804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:45.903971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:45.928171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce35a9-01c0-4076-9924-77418f4fae35 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:45.934215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1095/2159] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:45.945693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:45.947122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:45.982599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:45.984027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:45.984487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:45.984877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:45.985623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:45.987421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:46.000716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1065/2160] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:45:46.000877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:46.002182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:46.012091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e37b-8b21-4d71-85f2-4f530d00328d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:46.012241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acfd80-d261-439e-9a21-f0809b4dcb97 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:46.015840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28624ac1-be87-41c7-b490-c8d1fb7ffebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:46.016493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28624ac1-be87-41c7-b490-c8d1fb7ffebe service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:46.016809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28624ac1-be87-41c7-b490-c8d1fb7ffebe service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:46.017092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28624ac1-be87-41c7-b490-c8d1fb7ffebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:46.017913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:46.018565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:46.018880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:46.019159 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:46.019883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:46.021512 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:46.026299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28624ac1-be87-41c7-b490-c8d1fb7ffebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:46.027743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28624ac1-be87-41c7-b490-c8d1fb7ffebe service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:46.033653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1066/2161] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:46.038717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:46.042951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1096/2162] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:45:46.057078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bd785-9eac-44c7-b0c9-6a4a9dd14e20 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:46.062417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:46.064014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:46.064321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:46.064654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] PATH_INFO: `/v3/projects/b3f530bfb0ba455ab182c65ed3a6bcfc/users/0c6a54c404c84a2e912e6f7ebc42c815/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:46.080671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f530bfb0ba455ab182c65ed3a6bcfc, user_id=0c6a54c404c84a2e912e6f7ebc42c815, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:46.083877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:46.200109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1067/2163] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:46 2022] PUT /identity/v3/projects/b3f530bfb0ba455ab182c65ed3a6bcfc/users/0c6a54c404c84a2e912e6f7ebc42c815/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:46.211559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:46.211856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:46.212136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:46.242981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] MFA Rules not processed for user `0c6a54c404c84a2e912e6f7ebc42c815`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:46.244221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:46.341652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1097/2164] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:46.355149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:46.357221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:46.423906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306253c1-1e0a-438e-833e-66a4a1cbd3ba admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:46.428032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:46.428775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:46.429060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:46.429340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:46.439281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:46.440790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:46.449823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1068/2165] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:45:46.603072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:46.604891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:46.701438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ee435-4ad1-4e74-8f2a-b10f01bdc65e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:46.706742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:46.707616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:46.707906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:46.708188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] PATH_INFO: `/v3/projects/e78d5ab653cf4d1db63a7171a4e0b2b9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:46.709417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78d5ab653cf4d1db63a7171a4e0b2b9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:46.711449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:46.892501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78d5ab653cf4d1db63a7171a4e0b2b9', 'request_id': 'req-44947269-6423-4fa4-9358-3d152fbf42b4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:46.898687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78d5ab653cf4d1db63a7171a4e0b2b9', 'request_id': 'req-44947269-6423-4fa4-9358-3d152fbf42b4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:46.995501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78d5ab653cf4d1db63a7171a4e0b2b9', 'request_id': 'req-44947269-6423-4fa4-9358-3d152fbf42b4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.008214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] The token cache is being invalidate because project e78d5ab653cf4d1db63a7171a4e0b2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:47.008510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.009193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.011191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1098/2166] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:46 2022] DELETE /identity/v3/projects/e78d5ab653cf4d1db63a7171a4e0b2b9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:47.015250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:47.034860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e120-0bfb-4a73-82f9-a5a7e365be38 service glance] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:47.061443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:47.091624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:47.092716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:47.093450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:47.093797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] PATH_INFO: `/v3/users/790f49d59001495cbfa33783a4983ed1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:47.095001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=790f49d59001495cbfa33783a4983ed1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:47.096925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:47.370585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790f49d59001495cbfa33783a4983ed1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.382794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790f49d59001495cbfa33783a4983ed1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.389655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790f49d59001495cbfa33783a4983ed1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.630119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790f49d59001495cbfa33783a4983ed1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.630776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-76f4d5f4-930f-40e1-a1bd-ff563150e0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790f49d59001495cbfa33783a4983ed1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:47.637097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1069/2167] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:47 2022] DELETE /identity/v3/users/790f49d59001495cbfa33783a4983ed1 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:48.163852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:48.166337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:48.178330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:48.183089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44947269-6423-4fa4-9358-3d152fbf42b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:48.246712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:48.247707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:48.248175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:48.248660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] PATH_INFO: `/v3/projects/344fab8c44b6495792d747e2e65f61c3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:48.250820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344fab8c44b6495792d747e2e65f61c3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:48.253194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:48.431085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344fab8c44b6495792d747e2e65f61c3', 'request_id': 'req-626a72de-8ba0-4e94-b3ef-910d8b0a6438'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:48.438881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344fab8c44b6495792d747e2e65f61c3', 'request_id': 'req-626a72de-8ba0-4e94-b3ef-910d8b0a6438'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:48.564389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344fab8c44b6495792d747e2e65f61c3', 'request_id': 'req-626a72de-8ba0-4e94-b3ef-910d8b0a6438'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:48.577939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] The token cache is being invalidate because project 344fab8c44b6495792d747e2e65f61c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:48.578352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:48.579027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-626a72de-8ba0-4e94-b3ef-910d8b0a6438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:48.581204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1099/2168] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:48 2022] DELETE /identity/v3/projects/344fab8c44b6495792d747e2e65f61c3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:48.585773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908becb-0222-4f74-b343-ec852851445d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:48.586270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908becb-0222-4f74-b343-ec852851445d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:48.586460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908becb-0222-4f74-b343-ec852851445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:48.606664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1908becb-0222-4f74-b343-ec852851445d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:48.608325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1908becb-0222-4f74-b343-ec852851445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:48.693784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1070/2169] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:48.702311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:48.702688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:48.702934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:48.725056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:48.728182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:48.741710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1100/2170] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:48.745968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908becb-0222-4f74-b343-ec852851445d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:48.756736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908becb-0222-4f74-b343-ec852851445d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:48.761940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:48.762753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:48.763198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:48.763656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:48.764551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:45:48.765112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:48.766914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:48.782289 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1071/2171] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:45:48.786748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:48.813446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644740b-a094-4f81-8ef2-07909bcdade4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:48.823821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:48.824718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:48.825272 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:48.825655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:48.826959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:48.829984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:48.955353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1101/2172] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:48 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:48.959749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:48.975549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7cdc5-edf1-43ef-8d56-8194a63f0722 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.014077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:49.049916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.050615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.050903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.051185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.051897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.054190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.105786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.107741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:49.121995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132ba3e-6be6-4de3-9712-4e8f850c303c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.131132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.132155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.132537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.132914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] PATH_INFO: `/v3/users/0c769307ad5e4e5da6f811e81d194842` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.134624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c769307ad5e4e5da6f811e81d194842)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.136713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.184624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1072/2173] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:48 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:49.191839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.201602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e69d8-5e03-4b95-b8b0-3dde32732984 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.210887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.211624 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.211975 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.212317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.213101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.214787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.226409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1073/2174] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:45:49.230401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.240274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6676-e031-4b62-b25d-be460643372a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.244931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.245562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.245845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.246117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.246877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.248520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.258397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1074/2175] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:45:49.262869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.279145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3821-589c-4b04-be48-3b8ae887a1f4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.285433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e50b0-8618-4cdb-ab18-a6db4b49b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.286316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e50b0-8618-4cdb-ab18-a6db4b49b234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.286603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e50b0-8618-4cdb-ab18-a6db4b49b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.286883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e50b0-8618-4cdb-ab18-a6db4b49b234 admin admin] PATH_INFO: `/v3/projects/f4c53d1b0bd840c28b3c7084c7882eca/users/f207e38aacc344e79284ec33046d65b6/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.303773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e50b0-8618-4cdb-ab18-a6db4b49b234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c53d1b0bd840c28b3c7084c7882eca, user_id=f207e38aacc344e79284ec33046d65b6, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.306767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e50b0-8618-4cdb-ab18-a6db4b49b234 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.340171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c769307ad5e4e5da6f811e81d194842'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:49.348442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c769307ad5e4e5da6f811e81d194842'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:49.352498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:49.416196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1075/2176] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:49 2022] PUT /identity/v3/projects/f4c53d1b0bd840c28b3c7084c7882eca/users/f207e38aacc344e79284ec33046d65b6/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:49.428508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.428850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.429144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.456449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c769307ad5e4e5da6f811e81d194842'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:49.457289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c769307ad5e4e5da6f811e81d194842'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:49.457831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] MFA Rules not processed for user `f207e38aacc344e79284ec33046d65b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:49.458128 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d9bcd4-dda1-4147-acae-20a85db9a4be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c769307ad5e4e5da6f811e81d194842'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:49.459197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:49.491997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1102/2177] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:45:49 2022] DELETE /identity/v3/users/0c769307ad5e4e5da6f811e81d194842 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:49.496363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.496661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.496989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.521214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:49.522462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:49.541972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1076/2178] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:49.546706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.548592 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:49.596652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1103/2179] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:45:49.608248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.612718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:49.626037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796ecaf-25dc-48d5-85bd-ce37c8b341b2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.631208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.631876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.632168 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.632452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.633198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.635252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.654127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:49.684625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a990ad7-fd41-497a-af16-3bdd59ec0f15 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.692022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.692930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.693238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.693639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.709126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.710795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:49.721901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1104/2180] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:49.794685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1077/2181] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:49 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:49.799269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:49.810877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb7a7c-2e67-4de7-9a69-c382412bf74c service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:49.908366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:49.910827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:49.911141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:49.911557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:49.912719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:49.919155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.060520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1105/2182] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:49 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:50.066442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:50.094394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6845ca-906e-44e2-bec5-fad7aa25b0b7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:50.104356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:50.105145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.105879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.105879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.106431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:50.108065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.126515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1078/2183] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:50.127825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:50.140420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c54a1-891e-452a-a053-a7a4a946274a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:50.146481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60015-12ee-4a80-b7c0-2baa7e8644b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:50.148619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60015-12ee-4a80-b7c0-2baa7e8644b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.148619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60015-12ee-4a80-b7c0-2baa7e8644b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.148619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60015-12ee-4a80-b7c0-2baa7e8644b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.149338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa60015-12ee-4a80-b7c0-2baa7e8644b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:50.151131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa60015-12ee-4a80-b7c0-2baa7e8644b5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.161624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:50.173021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1106/2184] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:50.177664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:50.190061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c1ed0-1288-404a-ba16-404a75ad2e18 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:50.205473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:50.206388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.206674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.206950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] PATH_INFO: `/v3/projects/5c535bf6bbfe4501aad4e85b047eb667/users/1c9a74db21e54d8da864fcb7fd856990/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.234500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c535bf6bbfe4501aad4e85b047eb667, user_id=1c9a74db21e54d8da864fcb7fd856990, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:50.237491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.327926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1079/2185] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:50 2022] PUT /identity/v3/projects/5c535bf6bbfe4501aad4e85b047eb667/users/1c9a74db21e54d8da864fcb7fd856990/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:50.339772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.340071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.340361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.368629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] MFA Rules not processed for user `1c9a74db21e54d8da864fcb7fd856990`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:50.369884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:50.432429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:50.435591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:50.439631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1107/2186] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:50.446051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:50.525477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacaff6-394c-4969-97ac-e1d30a5e9048 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:50.529821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440efaa-4ea6-44c1-beb9-9323f84ea1cc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:50.530478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:50.531122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.531412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.531694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.532434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:50.534401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.535413 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:50.536948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.537247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.537535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] PATH_INFO: `/v3/projects/cd20523ecc3540afb3fa9662b805d8e8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.538907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd20523ecc3540afb3fa9662b805d8e8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:50.540907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.660350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1108/2187] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:45:50 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:45:50.661624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:50.702250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd20523ecc3540afb3fa9662b805d8e8', 'request_id': 'req-4b46d40b-3605-4114-82f7-6830af12033e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:50.744306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c361a7-edd5-4956-8fa5-ee32c02f2ac3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:50.807370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd20523ecc3540afb3fa9662b805d8e8', 'request_id': 'req-4b46d40b-3605-4114-82f7-6830af12033e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:50.823955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd20523ecc3540afb3fa9662b805d8e8', 'request_id': 'req-4b46d40b-3605-4114-82f7-6830af12033e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:50.830396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:50.831181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:50.831536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:50.831882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:50.832705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:50.835093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:50.845803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] The token cache is being invalidate because project cd20523ecc3540afb3fa9662b805d8e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:45:50.846279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:50.847031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:45:50.848722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1080/2188] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:45:50 2022] DELETE /identity/v3/projects/cd20523ecc3540afb3fa9662b805d8e8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:50.948011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:50.997622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1109/2189] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:45:50 2022] POST /identity/v3/users => generated 298 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:45:51.002876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:51.004437 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:51.031681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d40b-3605-4114-82f7-6830af12033e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:51.037461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:51.038249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:51.038610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:51.039115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:51.039965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:51.041794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:51.059201 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1081/2190] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:45:51.064596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:51.082910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc471c-0972-4afd-9ebd-2afbeb8dcb35 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:51.100604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:51.104667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:51.108427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:51.108427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] PATH_INFO: `/v3/projects/2808b5e14d0a45aab115aad3de6f74b0/users/0082a72fa30948c98f37fcea5d481122/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:51.128520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2808b5e14d0a45aab115aad3de6f74b0, user_id=0082a72fa30948c98f37fcea5d481122, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:51.134344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:51.271262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1110/2191] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:51 2022] PUT /identity/v3/projects/2808b5e14d0a45aab115aad3de6f74b0/users/0082a72fa30948c98f37fcea5d481122/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:45:51.271594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:51.301385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f9618-73ed-461e-9b8e-c42ee866e013 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:51.398276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:51.429216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c84f7-3bab-4bdf-950d-feaa8680c999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:51.431800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c84f7-3bab-4bdf-950d-feaa8680c999 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:51.431800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c84f7-3bab-4bdf-950d-feaa8680c999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:51.431800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c84f7-3bab-4bdf-950d-feaa8680c999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:51.431800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c84f7-3bab-4bdf-950d-feaa8680c999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:51.433327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c84f7-3bab-4bdf-950d-feaa8680c999 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:51.453121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1082/2192] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:45:51.457110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:51.469674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef334a7-bd13-49c2-8508-893e4508e6f2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:51.474784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:51.475718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:51.487040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:51.487040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] PATH_INFO: `/v3/projects/2808b5e14d0a45aab115aad3de6f74b0/users/0082a72fa30948c98f37fcea5d481122/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:51.487040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2808b5e14d0a45aab115aad3de6f74b0, user_id=0082a72fa30948c98f37fcea5d481122, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:51.487040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:51.493019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1111/2193] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:45:51 2022] PUT /identity/v3/projects/2808b5e14d0a45aab115aad3de6f74b0/users/0082a72fa30948c98f37fcea5d481122/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:45:51.503772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde17a2d-88aa-4df5-8388-c5f4065fc42c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:51.504069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde17a2d-88aa-4df5-8388-c5f4065fc42c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:51.504438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde17a2d-88aa-4df5-8388-c5f4065fc42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:51.536948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fde17a2d-88aa-4df5-8388-c5f4065fc42c None None] MFA Rules not processed for user `0082a72fa30948c98f37fcea5d481122`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:51.538216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fde17a2d-88aa-4df5-8388-c5f4065fc42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:51.615472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1083/2194] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:51.627366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:51.629288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:51.690584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0b35c-2a37-4b74-8dec-753d76f72638 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:51.694324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:51.695010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:51.695298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:51.695583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:51.697418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:51.759138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:51.760594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:51.770163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1112/2195] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:53.541655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:53.542115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:53.542248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:53.562193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:45:53.563600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:53.640279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1084/2196] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:45:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:45:53.683407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:53.689356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:53.723581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:45:53.767788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7785a-d6e0-4d04-aed1-623039b53fee service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:53.773121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:53.773779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:53.774069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:53.774353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:53.783430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:53.785097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:53.801126 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1113/2197] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:45:53.852662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:53.873337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e370d4-db7f-499c-abdf-3986ca93ad7c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:53.879044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:53.879972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:53.880342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:53.880726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:53.882695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:45:53.956858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:53.958387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:53.971368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1085/2198] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:45:57.108201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:45:57.119709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639149bf-4310-4839-9cd6-c5d4283232c4 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:45:57.125943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a998f-99ef-4d6d-ab0d-7ee5d511552f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:45:57.126569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a998f-99ef-4d6d-ab0d-7ee5d511552f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:45:57.126569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a998f-99ef-4d6d-ab0d-7ee5d511552f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:45:57.128214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a998f-99ef-4d6d-ab0d-7ee5d511552f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:45:57.137791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a998f-99ef-4d6d-ab0d-7ee5d511552f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:45:57.143230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a998f-99ef-4d6d-ab0d-7ee5d511552f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:45:57.152343 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1114/2199] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:45:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:46:02.665062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:02.676217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:02.788652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28f29b-0086-442d-a40e-7521e66d6aab service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:02.800850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:02.800850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:02.804122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:02.804122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] PATH_INFO: `/v3/users/f207e38aacc344e79284ec33046d65b6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:02.804737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f207e38aacc344e79284ec33046d65b6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:02.808021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:03.149638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f207e38aacc344e79284ec33046d65b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:03.168225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f207e38aacc344e79284ec33046d65b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:03.189770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f207e38aacc344e79284ec33046d65b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:03.260822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f207e38aacc344e79284ec33046d65b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:03.260822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f207e38aacc344e79284ec33046d65b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:03.293068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1086/2200] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:46:02 2022] DELETE /identity/v3/users/f207e38aacc344e79284ec33046d65b6 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:03.293604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:03.293604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:03.293604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:03.325039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:03.326426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:03.407537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1115/2201] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:03.421903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:03.424392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:03.559140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ac14b-4c28-4fab-a589-d75de2b855b9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:03.559140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:03.559140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:03.559140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:03.559636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:03.569580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:03.571365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:03.584653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1087/2202] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:04.912260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:04.914047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:04.933059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8239e-42ce-42ce-991b-120b23ec81d8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:04.945391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:04.947656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:04.949123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:04.949651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] PATH_INFO: `/v3/projects/f4c53d1b0bd840c28b3c7084c7882eca` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:04.952122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c53d1b0bd840c28b3c7084c7882eca)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:04.955531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:05.122428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c53d1b0bd840c28b3c7084c7882eca', 'request_id': 'req-de842945-7acc-45f2-a010-2922e5c57951'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.131803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c53d1b0bd840c28b3c7084c7882eca', 'request_id': 'req-de842945-7acc-45f2-a010-2922e5c57951'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.229415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c53d1b0bd840c28b3c7084c7882eca', 'request_id': 'req-de842945-7acc-45f2-a010-2922e5c57951'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.243910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] The token cache is being invalidate because project f4c53d1b0bd840c28b3c7084c7882eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:46:05.244215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.244920 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-de842945-7acc-45f2-a010-2922e5c57951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.250132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1116/2203] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:46:04 2022] DELETE /identity/v3/projects/f4c53d1b0bd840c28b3c7084c7882eca => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:05.252478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:05.265998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f75b6-78df-44a5-883f-713c6dfd407e service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:05.315684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:05.361324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:05.362232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:05.362543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:05.362824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] PATH_INFO: `/v3/users/1c9a74db21e54d8da864fcb7fd856990` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:05.364088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c9a74db21e54d8da864fcb7fd856990)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:05.366178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:05.830355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c9a74db21e54d8da864fcb7fd856990'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.843093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c9a74db21e54d8da864fcb7fd856990'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.849898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c9a74db21e54d8da864fcb7fd856990'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.949316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c9a74db21e54d8da864fcb7fd856990'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.949930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c9a74db21e54d8da864fcb7fd856990'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:05.956974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1088/2204] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:46:05 2022] DELETE /identity/v3/users/1c9a74db21e54d8da864fcb7fd856990 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:06.245987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:06.246521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:06.246626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:06.265996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] MFA Rules not processed for user `46d0d621460b4e5b877ef421d3503340`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:06.267227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:06.293096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:06.331019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1117/2205] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:06.625773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:06.627245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:06.703815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9b4d-f464-4776-a44b-a50ff806d057 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:06.708980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:06.709940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:06.710404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:06.710731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] PATH_INFO: `/v3/projects/5c535bf6bbfe4501aad4e85b047eb667` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:06.711948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c535bf6bbfe4501aad4e85b047eb667)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:06.716549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:06.850721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:06.853681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c535bf6bbfe4501aad4e85b047eb667', 'request_id': 'req-eb656378-9001-436d-b66f-5c38c4207398'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:06.969642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c535bf6bbfe4501aad4e85b047eb667', 'request_id': 'req-eb656378-9001-436d-b66f-5c38c4207398'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:06.975834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c535bf6bbfe4501aad4e85b047eb667', 'request_id': 'req-eb656378-9001-436d-b66f-5c38c4207398'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:06.987586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] The token cache is being invalidate because project 5c535bf6bbfe4501aad4e85b047eb667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:46:06.987879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:06.988519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:06.990465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1089/2206] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:46:06 2022] DELETE /identity/v3/projects/5c535bf6bbfe4501aad4e85b047eb667 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:06.994532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:07.005600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f573-8ad8-4150-875f-1e06f9dca173 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:07.060807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:07.061684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:07.061978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:07.062260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] PATH_INFO: `/v3/users/0082a72fa30948c98f37fcea5d481122` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:07.063453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0082a72fa30948c98f37fcea5d481122)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:07.065414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:07.333837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0082a72fa30948c98f37fcea5d481122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:07.340425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0082a72fa30948c98f37fcea5d481122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:07.529325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0082a72fa30948c98f37fcea5d481122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:07.530534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0082a72fa30948c98f37fcea5d481122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:07.531138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0082a72fa30948c98f37fcea5d481122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:07.549619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1118/2207] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:46:06 2022] DELETE /identity/v3/users/0082a72fa30948c98f37fcea5d481122 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:08.031095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.032537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:08.050193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb656378-9001-436d-b66f-5c38c4207398 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.116435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.117511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.117801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.118089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] PATH_INFO: `/v3/projects/2808b5e14d0a45aab115aad3de6f74b0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.119409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2808b5e14d0a45aab115aad3de6f74b0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.121363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.255388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2808b5e14d0a45aab115aad3de6f74b0', 'request_id': 'req-d64685f4-0fff-4ec1-ada1-b09e133ebd45'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:08.315781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.317949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:08.352857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2808b5e14d0a45aab115aad3de6f74b0', 'request_id': 'req-d64685f4-0fff-4ec1-ada1-b09e133ebd45'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:08.354749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2808b5e14d0a45aab115aad3de6f74b0', 'request_id': 'req-d64685f4-0fff-4ec1-ada1-b09e133ebd45'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:08.369734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] The token cache is being invalidate because project 2808b5e14d0a45aab115aad3de6f74b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:46:08.370196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:08.370868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d64685f4-0fff-4ec1-ada1-b09e133ebd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:08.376732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1090/2208] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:46:08 2022] DELETE /identity/v3/projects/2808b5e14d0a45aab115aad3de6f74b0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:08.382342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d5dfe-5edb-4f9e-b858-9024761be7f3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.382640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d5dfe-5edb-4f9e-b858-9024761be7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.382927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d5dfe-5edb-4f9e-b858-9024761be7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.387907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:08.407117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-781d5dfe-5edb-4f9e-b858-9024761be7f3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:08.407330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f96036-b7d3-41b8-affc-1cb0681eae3f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.408522 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-781d5dfe-5edb-4f9e-b858-9024761be7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:08.491198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.491900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.492190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.492471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.494340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:08.506136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.507793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.508562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1091/2209] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:08.518779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1119/2210] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:46:08.521695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.521996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.522280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.530155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:08.540825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:08.542183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:08.550884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1092/2211] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:08.555291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.565539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce707118-ddec-410f-8d63-b317f37cf6fc service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.570078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.570724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.571011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.571290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.572013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:46:08.572350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.573992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.586802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1120/2212] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:46:08.591092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.601419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0abc2a-d513-40c0-9230-dededf834c75 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.606302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.606959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.607246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.607581 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.608410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.610401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.733264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1093/2213] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:46:08 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:08.737273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.747193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3e8db-c56c-4e08-b99f-07b77b7bffa9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.806267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.806940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.807232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.807517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.808214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.810168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.912985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1121/2214] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:46:08 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:46:08.917232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.929518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2bc7a-c880-4f46-ad64-6d6d3b980a63 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.934279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.934989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.935237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.935520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.937112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.938731 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.951069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1094/2215] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:46:08.956301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:08.967268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8799eb3-e136-432e-95dc-0c645ad897b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:08.972077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e51096-5e18-4967-a0fb-7d3e1144afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:08.972815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51096-5e18-4967-a0fb-7d3e1144afbf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:08.973102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51096-5e18-4967-a0fb-7d3e1144afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:08.973393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51096-5e18-4967-a0fb-7d3e1144afbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:08.974130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51096-5e18-4967-a0fb-7d3e1144afbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:08.975733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51096-5e18-4967-a0fb-7d3e1144afbf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:08.986137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1122/2216] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:46:08.990160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:09.001006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f7463-cea8-4ebd-8336-3e3f1b2cc1f9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:09.006312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:09.006535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:09.006847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:09.007129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] PATH_INFO: `/v3/projects/8d52af3a60ac4c26bb3813a35a464197/users/644543f62dc74ecba883a1655da8e797/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:09.023032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d52af3a60ac4c26bb3813a35a464197, user_id=644543f62dc74ecba883a1655da8e797, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:09.025871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:09.118558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1095/2217] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:46:08 2022] PUT /identity/v3/projects/8d52af3a60ac4c26bb3813a35a464197/users/644543f62dc74ecba883a1655da8e797/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:09.130155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba340-79b6-4def-a2d3-ff927369de0b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:09.130554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba340-79b6-4def-a2d3-ff927369de0b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:09.130843 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ba340-79b6-4def-a2d3-ff927369de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:09.162107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a36ba340-79b6-4def-a2d3-ff927369de0b None None] MFA Rules not processed for user `644543f62dc74ecba883a1655da8e797`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:09.163364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a36ba340-79b6-4def-a2d3-ff927369de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:09.205827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:09.231779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1123/2218] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:09.243930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:09.245755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:09.327417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2729e-9722-4491-963f-2a2f73ff10a2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:09.331665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c8f79-c7db-46cf-9d60-ed8c2087e027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:09.332324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8f79-c7db-46cf-9d60-ed8c2087e027 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:09.332629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8f79-c7db-46cf-9d60-ed8c2087e027 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:09.332955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8f79-c7db-46cf-9d60-ed8c2087e027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:09.342693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c8f79-c7db-46cf-9d60-ed8c2087e027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:09.344175 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c8f79-c7db-46cf-9d60-ed8c2087e027 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:09.353872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1096/2219] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:14.765407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f2935-ffc3-48ee-a0a2-ada399573a9c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:14.765909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f2935-ffc3-48ee-a0a2-ada399573a9c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:14.766005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f2935-ffc3-48ee-a0a2-ada399573a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:14.784456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-be1f2935-ffc3-48ee-a0a2-ada399573a9c None None] MFA Rules not processed for user `f862a8b7b4534d1ba7d65b4688c32b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:14.785170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be1f2935-ffc3-48ee-a0a2-ada399573a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:14.843082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1124/2220] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:19.349417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:22.070261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b421c3-3577-409f-8152-394fa3381103 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.070915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b421c3-3577-409f-8152-394fa3381103 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.070915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b421c3-3577-409f-8152-394fa3381103 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.074129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1097/2221] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:46:22.082287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1114d-a661-44ee-98ce-80e317a1e462 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.084459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1114d-a661-44ee-98ce-80e317a1e462 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.084762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1114d-a661-44ee-98ce-80e317a1e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.106885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9de1114d-a661-44ee-98ce-80e317a1e462 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:22.108986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9de1114d-a661-44ee-98ce-80e317a1e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:22.192254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1125/2222] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:46:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:22.210485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b421c3-3577-409f-8152-394fa3381103 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:22.213354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e3b421c3-3577-409f-8152-394fa3381103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:22.301584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b421c3-3577-409f-8152-394fa3381103 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:22.301584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc03ac1-b935-40ef-b8c4-81c4cc4ca314 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:22.301787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc03ac1-b935-40ef-b8c4-81c4cc4ca314 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.301990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc03ac1-b935-40ef-b8c4-81c4cc4ca314 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.302238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc03ac1-b935-40ef-b8c4-81c4cc4ca314 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.314935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc03ac1-b935-40ef-b8c4-81c4cc4ca314 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:22.316445 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc03ac1-b935-40ef-b8c4-81c4cc4ca314 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:22.329666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1098/2223] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:46:22.699599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a007af2-78cb-4fbc-98c7-5a1ca5095a21 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.699985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a007af2-78cb-4fbc-98c7-5a1ca5095a21 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.700303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a007af2-78cb-4fbc-98c7-5a1ca5095a21 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.701928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1126/2224] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:46:22.708023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682285e-941b-4c48-ac50-f897cf2e8406 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.708346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682285e-941b-4c48-ac50-f897cf2e8406 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.708630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682285e-941b-4c48-ac50-f897cf2e8406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.727357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8682285e-941b-4c48-ac50-f897cf2e8406 None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:22.728568 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8682285e-941b-4c48-ac50-f897cf2e8406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:22.788438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1099/2225] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:46:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:22.801821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a007af2-78cb-4fbc-98c7-5a1ca5095a21 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:22.804611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a007af2-78cb-4fbc-98c7-5a1ca5095a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:22.823782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a007af2-78cb-4fbc-98c7-5a1ca5095a21 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:22.824968 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a529a2-3e59-4380-ae8c-abdccdbec195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:22.825699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a529a2-3e59-4380-ae8c-abdccdbec195 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.826085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a529a2-3e59-4380-ae8c-abdccdbec195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.826431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a529a2-3e59-4380-ae8c-abdccdbec195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.834381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a529a2-3e59-4380-ae8c-abdccdbec195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:22.835861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a529a2-3e59-4380-ae8c-abdccdbec195 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:22.843744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1127/2226] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:22.926485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660578d-4096-457e-9250-284c7489347a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.926878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660578d-4096-457e-9250-284c7489347a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.927261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660578d-4096-457e-9250-284c7489347a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.928867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1100/2227] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:46:22.936645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d7b0-4a4a-4336-938b-8679064aa8e3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:22.936978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d7b0-4a4a-4336-938b-8679064aa8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:22.937285 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d7b0-4a4a-4336-938b-8679064aa8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:22.955715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-04d5d7b0-4a4a-4336-938b-8679064aa8e3 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:22.956955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04d5d7b0-4a4a-4336-938b-8679064aa8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:22.966593 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1128/2228] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:46:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:22.983111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660578d-4096-457e-9250-284c7489347a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:22.985125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2660578d-4096-457e-9250-284c7489347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:22.999901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660578d-4096-457e-9250-284c7489347a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:23.003775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:23.004430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:23.004775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:23.005055 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:23.016407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:23.017897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:23.029771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1101/2229] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:32.950036 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=74 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:33.024827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=55 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:34.301818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:34.302276 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:34.302576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:34.327006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:34.328337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:34.397719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1129/2230] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:34.410940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:34.412470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:34.487775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975502c-6bfa-4ce9-afbd-03a36a3d4728 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:34.493616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc3a21-97df-4338-bc6f-6084776663be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:34.494277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc3a21-97df-4338-bc6f-6084776663be service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:34.494680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc3a21-97df-4338-bc6f-6084776663be service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:34.494967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc3a21-97df-4338-bc6f-6084776663be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:34.505431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc3a21-97df-4338-bc6f-6084776663be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:34.506422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc3a21-97df-4338-bc6f-6084776663be service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:34.523403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1102/2231] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:46:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:46:36.766951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:36.769196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:36.860557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab945-55b0-4407-9930-853251416d18 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:36.866527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:36.867544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:36.867837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:36.868120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] PATH_INFO: `/v3/users/0c6a54c404c84a2e912e6f7ebc42c815` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:36.869508 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6a54c404c84a2e912e6f7ebc42c815)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:36.871565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:37.624850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6a54c404c84a2e912e6f7ebc42c815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:37.632704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6a54c404c84a2e912e6f7ebc42c815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:37.754413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6a54c404c84a2e912e6f7ebc42c815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:37.755048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6a54c404c84a2e912e6f7ebc42c815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:37.755683 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6a54c404c84a2e912e6f7ebc42c815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:37.774552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1130/2232] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:46:36 2022] DELETE /identity/v3/users/0c6a54c404c84a2e912e6f7ebc42c815 => generated 0 bytes in 1011 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:37.779179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:37.779712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:37.779814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:37.798320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:46:37.801241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:37.864163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1103/2233] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:46:37.875045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:37.880501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:37.928595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:37.968648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fd42d-9e92-4d98-945a-3de00dfb6bfc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:37.972551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4e1b6-dda5-4bf2-9659-bb4d31738ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:37.973228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4e1b6-dda5-4bf2-9659-bb4d31738ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:37.973523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4e1b6-dda5-4bf2-9659-bb4d31738ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:37.973804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4e1b6-dda5-4bf2-9659-bb4d31738ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:37.991556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4e1b6-dda5-4bf2-9659-bb4d31738ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:37.993522 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4e1b6-dda5-4bf2-9659-bb4d31738ec4 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:38.006943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1131/2234] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:46:38.677918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:46:38.679630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:46:38.724879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad655fd-89a8-4aa4-92a4-d399d96beab4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:46:38.729816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:46:38.730805 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:46:38.731097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:46:38.732091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] PATH_INFO: `/v3/projects/b3f530bfb0ba455ab182c65ed3a6bcfc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:46:38.733384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f530bfb0ba455ab182c65ed3a6bcfc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:46:38.735711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:46:38.874600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f530bfb0ba455ab182c65ed3a6bcfc', 'request_id': 'req-4774c816-c14b-44cf-bd9b-1962150b66a7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:38.960333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f530bfb0ba455ab182c65ed3a6bcfc', 'request_id': 'req-4774c816-c14b-44cf-bd9b-1962150b66a7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:38.973501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f530bfb0ba455ab182c65ed3a6bcfc', 'request_id': 'req-4774c816-c14b-44cf-bd9b-1962150b66a7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:38.987110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] The token cache is being invalidate because project b3f530bfb0ba455ab182c65ed3a6bcfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:46:38.999893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:39.001878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:46:39.007735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1104/2235] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:46:38 2022] DELETE /identity/v3/projects/b3f530bfb0ba455ab182c65ed3a6bcfc => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:46:48.471974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=15 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:46:48.983467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:01.066642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:01.067099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:01.067217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:01.085914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:01.087117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:01.141326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1132/2236] 10.222.0.47 () {58 vars in 967 bytes} [Mon Jul 18 10:47:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:01.164117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:01.165546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:01.235401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774c816-c14b-44cf-bd9b-1962150b66a7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:01.240442 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380375b-d284-47c2-bf4e-61a28710b452 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:01.241127 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380375b-d284-47c2-bf4e-61a28710b452 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:01.241419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380375b-d284-47c2-bf4e-61a28710b452 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:01.241699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380375b-d284-47c2-bf4e-61a28710b452 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:01.250110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380375b-d284-47c2-bf4e-61a28710b452 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:01.251817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380375b-d284-47c2-bf4e-61a28710b452 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:01.260284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1105/2237] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:47:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:47:04.585260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:04.586934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:04.651887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d87bf0-c279-44d2-a722-52b9c344d7df None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:04.657032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:04.657901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:04.658192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:04.658472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] PATH_INFO: `/v3/users/644543f62dc74ecba883a1655da8e797` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:04.659678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644543f62dc74ecba883a1655da8e797)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:04.661644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:04.902914 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644543f62dc74ecba883a1655da8e797'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:04.910351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644543f62dc74ecba883a1655da8e797'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:05.042305 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644543f62dc74ecba883a1655da8e797'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:05.043602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644543f62dc74ecba883a1655da8e797'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:05.044398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644543f62dc74ecba883a1655da8e797'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:05.067637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1133/2238] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:04 2022] DELETE /identity/v3/users/644543f62dc74ecba883a1655da8e797 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:05.069529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:05.070073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:05.070291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:05.088885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:05.090664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:05.158746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1106/2239] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:47:05.169698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:05.171083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:05.199457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:05.237466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a123fd-7ae6-4bb3-afab-9f43ed2063b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:05.246564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879244-a4f5-441c-b01a-0050c29fa18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:05.246564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879244-a4f5-441c-b01a-0050c29fa18d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:05.246564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879244-a4f5-441c-b01a-0050c29fa18d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:05.246564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879244-a4f5-441c-b01a-0050c29fa18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:05.251622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879244-a4f5-441c-b01a-0050c29fa18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:05.253105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879244-a4f5-441c-b01a-0050c29fa18d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:05.264362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1134/2240] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:05.743995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:05.745548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:05.759265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71274-650b-4faa-9f89-8471caf22373 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:05.765423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:05.766292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:05.766574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:05.766847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] PATH_INFO: `/v3/projects/8d52af3a60ac4c26bb3813a35a464197` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:05.768057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d52af3a60ac4c26bb3813a35a464197)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:05.770023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:05.950781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d52af3a60ac4c26bb3813a35a464197', 'request_id': 'req-70b090ad-ec08-4169-9559-017bf2b3b307'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:06.070393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d52af3a60ac4c26bb3813a35a464197', 'request_id': 'req-70b090ad-ec08-4169-9559-017bf2b3b307'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:06.076672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d52af3a60ac4c26bb3813a35a464197', 'request_id': 'req-70b090ad-ec08-4169-9559-017bf2b3b307'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:06.088542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] The token cache is being invalidate because project 8d52af3a60ac4c26bb3813a35a464197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:06.088941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:06.096818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70b090ad-ec08-4169-9559-017bf2b3b307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:06.096818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1107/2241] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:05 2022] DELETE /identity/v3/projects/8d52af3a60ac4c26bb3813a35a464197 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:06.097018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.097427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.097516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.116034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:06.117286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:06.187826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1135/2242] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:06.196075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a37961-e515-4d00-b082-cb9758072463 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.196379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a37961-e515-4d00-b082-cb9758072463 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.196668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a37961-e515-4d00-b082-cb9758072463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.217005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-98a37961-e515-4d00-b082-cb9758072463 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:06.218380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-98a37961-e515-4d00-b082-cb9758072463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:06.226967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1108/2243] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:06.231792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.242140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acd76d-0cee-47d2-aa5b-0e4823e9a67d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.247288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.248069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.248562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.248947 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.249769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:47:06.250156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.252235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.270129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1136/2244] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:47:06.271767 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a37961-e515-4d00-b082-cb9758072463 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.281756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a37961-e515-4d00-b082-cb9758072463 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.286593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.288350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.288350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.288350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.288707 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.290627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.294774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:06.432410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1109/2245] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:06 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:06.436535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.446619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318cb31-4e58-47bc-b9ee-194976c24b7b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.505892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.506870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.507163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.507452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.508227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.510189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.642588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1137/2246] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:06 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:06.646680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.657301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f3df-b6e8-4c12-a949-602fd90f8379 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.662370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.663041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.663327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.663607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.664343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.665978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.676219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1110/2247] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:06.680205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.691010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b702-cd27-4cc7-8b24-490ffbcfe2ff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.695717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5d3f6-449f-4549-8bce-b09d39144b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.696354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5d3f6-449f-4549-8bce-b09d39144b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.696638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5d3f6-449f-4549-8bce-b09d39144b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.696974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5d3f6-449f-4549-8bce-b09d39144b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.697719 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5d3f6-449f-4549-8bce-b09d39144b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.700183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5d3f6-449f-4549-8bce-b09d39144b53 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.710351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1138/2248] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:06.714399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:06.724340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edaf7-ecdc-4dba-acd9-4d33cddd8dd4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:06.728870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:06.729734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.730018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.730461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] PATH_INFO: `/v3/projects/97b79ca700b4425ca0c6aa8a346e7eeb/users/686a36a7925f4a4c95d821c644628dfb/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.744903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b79ca700b4425ca0c6aa8a346e7eeb, user_id=686a36a7925f4a4c95d821c644628dfb, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:06.748197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:06.871987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1111/2249] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:06 2022] PUT /identity/v3/projects/97b79ca700b4425ca0c6aa8a346e7eeb/users/686a36a7925f4a4c95d821c644628dfb/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:06.939823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9be59c-b6bc-4220-8eb8-376ac5b95ed3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:06.940042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9be59c-b6bc-4220-8eb8-376ac5b95ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:06.940406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9be59c-b6bc-4220-8eb8-376ac5b95ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:06.968407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fa9be59c-b6bc-4220-8eb8-376ac5b95ed3 None None] MFA Rules not processed for user `686a36a7925f4a4c95d821c644628dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:06.969676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fa9be59c-b6bc-4220-8eb8-376ac5b95ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:06.990194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:07.036306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1139/2250] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:07.048039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:07.049828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:07.107597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25567-b818-45d0-a731-1499c5ea916c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:07.111548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e25f4-6076-4462-8190-07835cad980f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:07.112223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e25f4-6076-4462-8190-07835cad980f service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:07.112517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e25f4-6076-4462-8190-07835cad980f service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:07.112851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e25f4-6076-4462-8190-07835cad980f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:07.121933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e25f4-6076-4462-8190-07835cad980f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:07.123368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e25f4-6076-4462-8190-07835cad980f service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:07.132171 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1112/2251] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:10.085506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a47ae3-e71e-4034-a700-74f87cfc6765 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:10.086062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a47ae3-e71e-4034-a700-74f87cfc6765 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:10.086062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a47ae3-e71e-4034-a700-74f87cfc6765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:10.104013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-50a47ae3-e71e-4034-a700-74f87cfc6765 None None] MFA Rules not processed for user `686a36a7925f4a4c95d821c644628dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:10.106023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-50a47ae3-e71e-4034-a700-74f87cfc6765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:10.113346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1140/2252] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:17.128185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:20.101833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=20 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:26.437219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:26.437759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:26.437759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:26.455424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] MFA Rules not processed for user `b86d6011a3f04ad0bf5b8be696c789f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:26.456797 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:26.515325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1113/2253] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:36.507823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=25 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:40.039885 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556912d-1ecd-48dc-adce-7009dfbb49ba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:40.040836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556912d-1ecd-48dc-adce-7009dfbb49ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:40.041328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556912d-1ecd-48dc-adce-7009dfbb49ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:40.061453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a556912d-1ecd-48dc-adce-7009dfbb49ba None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:40.062081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a556912d-1ecd-48dc-adce-7009dfbb49ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:40.130009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1141/2254] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:40.146411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:40.148510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:40.229923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84da77-3745-46c8-8cae-304f4b18ca47 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:40.235402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:40.236052 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:40.236343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:40.236620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:40.245675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:40.247268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:40.258061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1114/2255] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:47:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:47:41.282674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e7171-f976-4edb-9a2c-f4d3495d1eb8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:41.283157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e7171-f976-4edb-9a2c-f4d3495d1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:41.283326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e7171-f976-4edb-9a2c-f4d3495d1eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:41.301403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5f1e7171-f976-4edb-9a2c-f4d3495d1eb8 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:41.302625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f1e7171-f976-4edb-9a2c-f4d3495d1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:41.310477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1142/2256] 10.222.0.47 () {58 vars in 967 bytes} [Mon Jul 18 10:47:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:41.322950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:41.334048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2c1b-d486-4252-9c6f-89b09df1fe15 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:41.339257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:41.339901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:41.340207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:41.340486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:41.348973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:41.350570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:41.359021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1115/2257] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:47:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:47:42.120503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95014f69-f2bd-416c-9af8-58c3cf61a390 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:42.121282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95014f69-f2bd-416c-9af8-58c3cf61a390 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:42.121282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95014f69-f2bd-416c-9af8-58c3cf61a390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:42.139378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-95014f69-f2bd-416c-9af8-58c3cf61a390 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:42.140596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-95014f69-f2bd-416c-9af8-58c3cf61a390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:42.147889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1143/2258] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:47:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:42.160410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:42.171949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f38835-48cc-4615-9fb6-a6932b958a77 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:42.177193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:42.177833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:42.178121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:42.178403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:42.186741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:42.188311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:42.196304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1116/2259] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:47:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:47:42.849605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:42.850102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:42.850188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:42.868274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:42.870438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:42.877839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1144/2260] 10.222.0.47 () {58 vars in 967 bytes} [Mon Jul 18 10:47:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:42.890951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:42.901761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413be20-e3ec-4937-af1b-df746f842a0a service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:42.906742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c472f0b-cdfa-4991-96d6-dcd0219a0c4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:42.907384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f0b-cdfa-4991-96d6-dcd0219a0c4a service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:42.907668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f0b-cdfa-4991-96d6-dcd0219a0c4a service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:42.907944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f0b-cdfa-4991-96d6-dcd0219a0c4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:42.916358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c472f0b-cdfa-4991-96d6-dcd0219a0c4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:42.918016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c472f0b-cdfa-4991-96d6-dcd0219a0c4a service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:42.926144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1117/2261] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:47:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:47:45.650148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:45.728259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b6030-13f8-400f-b38d-fe2d13da0a1f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:45.733915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:45.735161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:45.735644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:45.736121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] PATH_INFO: `/v3/users/686a36a7925f4a4c95d821c644628dfb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:45.737673 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=686a36a7925f4a4c95d821c644628dfb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:45.739757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:46.146735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686a36a7925f4a4c95d821c644628dfb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:46.153896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686a36a7925f4a4c95d821c644628dfb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:46.243917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686a36a7925f4a4c95d821c644628dfb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:46.245368 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686a36a7925f4a4c95d821c644628dfb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:46.245989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686a36a7925f4a4c95d821c644628dfb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:46.269670 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1145/2262] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:45 2022] DELETE /identity/v3/users/686a36a7925f4a4c95d821c644628dfb => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:46.274198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:46.274637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:46.274814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:46.300892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:46.300892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:46.362045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1118/2263] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:46.371796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:46.373432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:46.381270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:46.436059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08ee38-8a9a-428e-8ed6-93a598a07972 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:46.439794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa69d9-d538-4776-a7eb-bc65f607a7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:46.440454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa69d9-d538-4776-a7eb-bc65f607a7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:46.440778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa69d9-d538-4776-a7eb-bc65f607a7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:46.441057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa69d9-d538-4776-a7eb-bc65f607a7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:46.450357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa69d9-d538-4776-a7eb-bc65f607a7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:46.451789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa69d9-d538-4776-a7eb-bc65f607a7b3 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:46.462204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1146/2264] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:47:46.757260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:46.758672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:46.771864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75deff-802a-4106-936a-4d60dfc1305f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:46.778006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:46.778881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:46.779343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:46.779628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] PATH_INFO: `/v3/projects/97b79ca700b4425ca0c6aa8a346e7eeb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:46.780930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b79ca700b4425ca0c6aa8a346e7eeb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:46.783327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:46.935268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:46.938429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b79ca700b4425ca0c6aa8a346e7eeb', 'request_id': 'req-8b33376a-2963-45b4-82e2-56140160b2bc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:47.055853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b79ca700b4425ca0c6aa8a346e7eeb', 'request_id': 'req-8b33376a-2963-45b4-82e2-56140160b2bc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:47.063012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b79ca700b4425ca0c6aa8a346e7eeb', 'request_id': 'req-8b33376a-2963-45b4-82e2-56140160b2bc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:47.074162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] The token cache is being invalidate because project 97b79ca700b4425ca0c6aa8a346e7eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:47.074452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:47.075098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:47.076741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1119/2265] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:46 2022] DELETE /identity/v3/projects/97b79ca700b4425ca0c6aa8a346e7eeb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:55.841892 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:55.842329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:55.842507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:55.861393 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] MFA Rules not processed for user `46d0d621460b4e5b877ef421d3503340`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:55.862722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:55.923187 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1147/2266] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:56.818268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:56.819752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:56.891176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33376a-2963-45b4-82e2-56140160b2bc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:56.896035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:56.896957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:56.897244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:56.897524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] PATH_INFO: `/v3/users/f862a8b7b4534d1ba7d65b4688c32b92` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:56.898753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f862a8b7b4534d1ba7d65b4688c32b92)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:56.901812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:56.998231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:57.000174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:57.067747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed4663-5e2d-4b2f-8203-421ebe3abf22 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:57.071365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:57.072180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:57.072478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:57.072790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:57.074973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:57.150074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f862a8b7b4534d1ba7d65b4688c32b92'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.154623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:57.159978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:57.161536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:57.165258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f862a8b7b4534d1ba7d65b4688c32b92'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.175615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f862a8b7b4534d1ba7d65b4688c32b92'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.176887 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1148/2267] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:57.312957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f862a8b7b4534d1ba7d65b4688c32b92'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.314701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33bc854a-f3ed-472d-a43e-b0dab18153b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f862a8b7b4534d1ba7d65b4688c32b92'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.322680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1120/2268] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:47:56 2022] DELETE /identity/v3/users/f862a8b7b4534d1ba7d65b4688c32b92 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:57.609428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:57.620701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47039a-4a12-499a-85d9-5ad5117abe1a service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:57.682253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:57.683117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:57.683426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:57.683703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] PATH_INFO: `/v3/projects/bac9bcfdd4ef4a6d87f593bde0c352c9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:57.685034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac9bcfdd4ef4a6d87f593bde0c352c9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:57.686973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:57.700020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:57.700342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:57.700627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:57.722788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] MFA Rules not processed for user `df7a3101955b481f9a582168731ae76c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:57.724325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:57.775414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:57.784152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1121/2269] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:47:57.818586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:57.819990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:57.872423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac9bcfdd4ef4a6d87f593bde0c352c9', 'request_id': 'req-edc45d94-9b17-4673-85eb-a0922ed36dc6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.878677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac9bcfdd4ef4a6d87f593bde0c352c9', 'request_id': 'req-edc45d94-9b17-4673-85eb-a0922ed36dc6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.883063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ee58-5337-4aad-b560-f593579c58bc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:57.934847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:57.935503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:57.935828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:57.936074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:57.937930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:57.983851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac9bcfdd4ef4a6d87f593bde0c352c9', 'request_id': 'req-edc45d94-9b17-4673-85eb-a0922ed36dc6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.997016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] The token cache is being invalidate because project bac9bcfdd4ef4a6d87f593bde0c352c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:47:57.997313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.997949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc45d94-9b17-4673-85eb-a0922ed36dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:47:57.999496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1149/2270] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:47:57 2022] DELETE /identity/v3/projects/bac9bcfdd4ef4a6d87f593bde0c352c9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:58.004316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.004618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.007542 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.011468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.012953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf47eca-d6bf-4265-85d9-e7d53eae885a service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.023638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:58.024865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1122/2271] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:47:58.025571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:58.085682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1150/2272] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:47:58.094021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.094315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.094593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.112609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:58.113688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:58.123650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1123/2273] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:58.128164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.138942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408c7f-afb9-4158-911d-3d475b995cbd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.143681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.144337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.144627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.144950 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.145680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:47:58.146009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.147586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.163534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1151/2274] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:47:58.164753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.174702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9b567-a724-4089-9774-41bcec06111a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.179483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.180157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.180585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.180901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.181693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.183903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.329794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1124/2275] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:47:58 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:47:58.334101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.345363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83400-9e13-44dd-ac6a-1fe02973d47d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.355295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:58.406080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.406682 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.406974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.407257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.407969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.409942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.553661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1152/2276] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:47:58 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:47:58.558440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.570877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ca49f-304f-4968-bee1-0db78877f0d2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.576715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.577380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.577674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.577952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.578691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.580540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.591389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1125/2277] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:58.595506 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.606127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51551a44-786b-4738-a70d-83b357aa872a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.611450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a6d9f-9e38-4b5e-9e60-a9187a2de7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.612123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6d9f-9e38-4b5e-9e60-a9187a2de7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.612409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6d9f-9e38-4b5e-9e60-a9187a2de7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.612712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6d9f-9e38-4b5e-9e60-a9187a2de7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.613461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a6d9f-9e38-4b5e-9e60-a9187a2de7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.615127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a6d9f-9e38-4b5e-9e60-a9187a2de7f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.626293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1153/2278] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:47:58.630462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.641545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460df9c-0a4a-40b9-8997-eda6d8a15fc5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.646146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.651373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.651373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.651373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] PATH_INFO: `/v3/projects/6d89e2b7a84b4fb6ba7060088b60f6ae/users/3cd9bc14a25846189a81c3a6845fd551/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.659670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:47:58.662627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d89e2b7a84b4fb6ba7060088b60f6ae, user_id=3cd9bc14a25846189a81c3a6845fd551, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.665285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:58.792236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1126/2279] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:47:58 2022] PUT /identity/v3/projects/6d89e2b7a84b4fb6ba7060088b60f6ae/users/3cd9bc14a25846189a81c3a6845fd551/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:47:58.803910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86362b-b9eb-4a94-916a-06b624963934 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.804091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86362b-b9eb-4a94-916a-06b624963934 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.804358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86362b-b9eb-4a94-916a-06b624963934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.823199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5f86362b-b9eb-4a94-916a-06b624963934 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:47:58.824418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f86362b-b9eb-4a94-916a-06b624963934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:58.899232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1154/2280] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:47:58.909817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:47:58.911626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:47:58.975871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e412f-9db6-4037-9c45-dc635042703e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:47:58.980503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:47:58.981183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:47:58.981489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:47:58.981785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:47:58.991468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:47:58.992931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:47:59.003193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1127/2281] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:06.693613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:06.694482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:06.694928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:06.727478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] MFA Rules not processed for user `3cd9bc14a25846189a81c3a6845fd551`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:06.728934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:06.812733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1155/2282] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:06.816840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:06.827721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111677d-4535-4c59-b727-4c02f2be2a9b service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:06.833608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:06.834275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:06.834562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:06.834844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:06.835580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:06.841464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:06.940023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1128/2283] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:06 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:06.944025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:06.956580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505270-d8ac-40a8-9817-e862ea05b488 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:06.982437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:07.032791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:07.033482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:07.033777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:07.034064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:07.034916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:07.041413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.130100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1156/2284] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:06 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:07.134706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:07.145237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bba26c-8d04-4486-b097-eccc1162ab9d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:07.150099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:07.150760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:07.151041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:07.151317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:07.152124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:07.153741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.164768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1129/2285] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:07.170058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:07.180668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3366b0-8d11-4fcc-939d-4bf5121d6a63 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:07.186463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:07.187382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:07.187668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:07.187946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] PATH_INFO: `/v3/projects/a0f8905c13a84c4ba3271019fdf189d2/users/abbd7c912b5848feb83c7335852006de/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:07.203265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f8905c13a84c4ba3271019fdf189d2, user_id=abbd7c912b5848feb83c7335852006de, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:07.206225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.299515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1157/2286] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:07 2022] PUT /identity/v3/projects/a0f8905c13a84c4ba3271019fdf189d2/users/abbd7c912b5848feb83c7335852006de/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:07.303780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:07.315708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286c02c-95b4-4682-8474-d1f69526520f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:07.381616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:07.384876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:07.384876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:07.384876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:07.384876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:07.386030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.396997 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1130/2287] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:07.401199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:07.421118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0687-3bde-40a6-a664-1185be40aa0f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:07.426339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75804d55-a7a3-44fd-af51-99928e01d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:07.427303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804d55-a7a3-44fd-af51-99928e01d7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:07.427658 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804d55-a7a3-44fd-af51-99928e01d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:07.428004 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804d55-a7a3-44fd-af51-99928e01d7a5 admin admin] PATH_INFO: `/v3/projects/a0f8905c13a84c4ba3271019fdf189d2/users/abbd7c912b5848feb83c7335852006de/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:07.430079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75804d55-a7a3-44fd-af51-99928e01d7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f8905c13a84c4ba3271019fdf189d2, user_id=abbd7c912b5848feb83c7335852006de, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:07.433560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75804d55-a7a3-44fd-af51-99928e01d7a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:07.444378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1158/2288] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:07 2022] PUT /identity/v3/projects/a0f8905c13a84c4ba3271019fdf189d2/users/abbd7c912b5848feb83c7335852006de/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:09.912739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:09.912739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:09.979980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:10.000298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03b21-60e2-4095-976a-fe57aede1deb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:10.079867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:10.080546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:10.081019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:10.081305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:10.083276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:10.158338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:10.160422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:10.172524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1131/2289] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:15.506729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3580-214a-4c96-8e1b-e8efce6830b9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:15.507257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3580-214a-4c96-8e1b-e8efce6830b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:15.507339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3580-214a-4c96-8e1b-e8efce6830b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:15.551729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-943d3580-214a-4c96-8e1b-e8efce6830b9 None None] MFA Rules not processed for user `abbd7c912b5848feb83c7335852006de`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:15.553114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-943d3580-214a-4c96-8e1b-e8efce6830b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:15.656112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1159/2290] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:15.670631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:15.672112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:15.691014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581816e3-6df3-45be-bfee-8b47ddb74635 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:15.698023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:15.698970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:15.699437 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:15.699889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:15.762081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:15.763721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:15.776312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1132/2291] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:19.864901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832544f2-d785-4d47-8fa6-ae030b448a3c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:19.865435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832544f2-d785-4d47-8fa6-ae030b448a3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:19.865717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832544f2-d785-4d47-8fa6-ae030b448a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:19.885907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-832544f2-d785-4d47-8fa6-ae030b448a3c None None] MFA Rules not processed for user `3cd9bc14a25846189a81c3a6845fd551`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:19.887686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-832544f2-d785-4d47-8fa6-ae030b448a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:19.899902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1160/2292] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:21.814580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:21.826736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1171-a2a0-4ae5-854a-ec9505be9c16 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:21.833153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:21.834135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:21.834423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:21.834705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:21.836670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:21.870252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:21.912193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:21.916377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:21.924324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1133/2293] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:27.174116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.174599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.181971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.192891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] MFA Rules not processed for user `17e0c6def385441bbdde18c63b7d99e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:27.194354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.201890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1161/2294] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:27.541524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:27.600542 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbe2d6-0b9f-4909-a428-7f71a571ad04 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:27.606075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:27.606850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:27.607156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:27.607438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:27.609629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:27.621371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:27.622801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:27.633235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1134/2295] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:48:28.137637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:28.139106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:28.191338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:28.215209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c50de42-63bb-4808-99a6-32e43f8c4277 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:28.220235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:28.220925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:28.221210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:28.221489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:28.223235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:28.237982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:28.239598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:28.251282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1162/2296] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:32.341545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:32.343008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:32.355129 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674efd-603e-4b4a-ae40-fd03759028e5 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:32.359199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:32.359933 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:32.360291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:32.360671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:32.362600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:32.374571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:32.376097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:32.385828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1135/2297] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:37.080072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:37.090376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad62254-bb35-441a-8552-545ae3c70c65 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:37.094188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:37.094835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:37.095125 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:37.095409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:37.097214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:37.107093 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:37.108598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:37.116662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1163/2298] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:37.533609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:37.547295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d1800-0916-49c8-bc76-81719483a55e service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:37.552952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:37.553592 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:37.553875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:37.554178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:37.556002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:37.568227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:37.569842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:37.580593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1136/2299] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:39.875342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:39.883976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89b628-14ac-478c-8776-7d40695d8c6d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:39.888493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:39.889175 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:39.889459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:39.889737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:39.891734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:39.965371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:39.966814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:39.977193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1164/2300] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:45.227974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:45.239810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21344d7-ecf3-4fac-afe7-c10c0ee49a88 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:45.245842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:45.246576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.246866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.247150 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.249027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.272658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:45.275259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:45.287087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1137/2301] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:45.432853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:45.434385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:45.446991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aea8c1-1c16-4c49-bddc-ac2e391f2bb8 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:45.452391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:45.453538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:45.454280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:45.454565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] PATH_INFO: `/v3/users/b86d6011a3f04ad0bf5b8be696c789f2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:45.455886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86d6011a3f04ad0bf5b8be696c789f2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:45.458047 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:45.745557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86d6011a3f04ad0bf5b8be696c789f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.753385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86d6011a3f04ad0bf5b8be696c789f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.868261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86d6011a3f04ad0bf5b8be696c789f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.868930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86d6011a3f04ad0bf5b8be696c789f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.869563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00118285-0e1c-4486-9f1b-6a5f3e186dcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86d6011a3f04ad0bf5b8be696c789f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:45.897149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1165/2302] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:48:45 2022] DELETE /identity/v3/users/b86d6011a3f04ad0bf5b8be696c789f2 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:46.207742 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.221074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5386e-01f1-4c25-8ce1-b4bd7a190804 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.283364 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.286929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.286929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.286929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] PATH_INFO: `/v3/projects/b0e325a6638442b9a2a929a6525f7c2e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.286929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e325a6638442b9a2a929a6525f7c2e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.288593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.441033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:46.460621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e325a6638442b9a2a929a6525f7c2e', 'request_id': 'req-51d91db4-149a-4c3a-bebd-2332234bbf1e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.622467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e325a6638442b9a2a929a6525f7c2e', 'request_id': 'req-51d91db4-149a-4c3a-bebd-2332234bbf1e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.628849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e325a6638442b9a2a929a6525f7c2e', 'request_id': 'req-51d91db4-149a-4c3a-bebd-2332234bbf1e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.641516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] The token cache is being invalidate because project b0e325a6638442b9a2a929a6525f7c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:48:46.641809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.642471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-51d91db4-149a-4c3a-bebd-2332234bbf1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:48:46.644061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1138/2303] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:48:46 2022] DELETE /identity/v3/projects/b0e325a6638442b9a2a929a6525f7c2e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:46.649335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.649845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.650073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.668520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:46.671461 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.691344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:46.731929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1166/2304] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:46.740034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.740339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.740626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.759792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:46.761056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.771121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1139/2305] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:46.775603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.786469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab961f-1ef6-447f-ac57-64b3090413d3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.792645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.793438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.802350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.802350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.802350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:48:46.802350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.802350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.813701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1167/2306] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:48:46.814788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.817695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.819692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.824765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7ae4-dce0-43d3-8c4c-1aba68522e4b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.830173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.830850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.831137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.831462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.832140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.834115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.902606 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e625174-4b14-489c-a464-cbef91d0422a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:46.908451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:46.909171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:46.909489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:46.909773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:46.911703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:46.924536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:46.926189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:46.937618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1168/2307] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:46.984674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1140/2308] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:46.988954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:46.999391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e146d13-946c-413b-9007-56ae569e97f5 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.065009 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.065763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.066152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.066558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.067382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.069473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.418959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1169/2309] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:46 2022] POST /identity/v3/users => generated 283 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:47.426113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.435333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a102f-9afe-4832-a760-2e76723a718a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.440789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.441477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.441790 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.442075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.442812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.444415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.461843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1141/2310] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:47.465704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.479731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b07d83-333d-4df3-9f4b-26d4306cb4bf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.485302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106704b4-d5e4-4cde-90f1-00c9c2b7db33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.486090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106704b4-d5e4-4cde-90f1-00c9c2b7db33 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.486388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106704b4-d5e4-4cde-90f1-00c9c2b7db33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.486676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106704b4-d5e4-4cde-90f1-00c9c2b7db33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.487479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106704b4-d5e4-4cde-90f1-00c9c2b7db33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.489355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106704b4-d5e4-4cde-90f1-00c9c2b7db33 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.504674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1170/2311] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:47.505915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:47.520212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae392de1-6825-4558-b8e2-975bb802e8d2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:47.527187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:47.528093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.528383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.528666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] PATH_INFO: `/v3/projects/03cc0e9fc7944bfdb9bf0c4d8d48bf66/users/d7df1a88e8fa4dabadcd4a85ee962e1c/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.553948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cc0e9fc7944bfdb9bf0c4d8d48bf66, user_id=d7df1a88e8fa4dabadcd4a85ee962e1c, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:47.556992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:47.914825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1142/2312] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:47 2022] PUT /identity/v3/projects/03cc0e9fc7944bfdb9bf0c4d8d48bf66/users/d7df1a88e8fa4dabadcd4a85ee962e1c/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:47.925856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:47.926356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:47.927473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:47.954769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] MFA Rules not processed for user `d7df1a88e8fa4dabadcd4a85ee962e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:47.956003 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.020412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1171/2313] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:48:48.025728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.087682 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fa3bc-64cc-4771-aea6-2f12e7c270f1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.092311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.093008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.093314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.093589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.094279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.096184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.255513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1143/2314] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:48:48 2022] POST /identity/v3/projects => generated 363 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:48:48.259781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.270902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105ee6e-c9db-4574-ae90-5ad3d2293e62 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.326599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.327267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.327558 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.327837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.328549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.330815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.499971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1172/2315] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:48:48 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:48:48.503697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.514229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adb1ae-d9c8-450b-bede-9ed2db1163bc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.518925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.519582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.520915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.520915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.521104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.522521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.534686 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1144/2316] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:48.539460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.549574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d43fa7-46bf-42ec-8329-025b72cd000a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.554867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.555906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.556193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.556471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] PATH_INFO: `/v3/projects/86cb5d6daebc439293f0f430a6235ba5/users/095fb5105a82489ea5d6485181018270/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.571033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cb5d6daebc439293f0f430a6235ba5, user_id=095fb5105a82489ea5d6485181018270, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.573710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.683672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:48.690756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1173/2317] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:48 2022] PUT /identity/v3/projects/86cb5d6daebc439293f0f430a6235ba5/users/095fb5105a82489ea5d6485181018270/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:48.694498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.705135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f234b3-9f35-4e7b-be39-ac856fe2b1bb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.738799 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:48.765274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204eec7-0ef0-460a-9002-a192588fb721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.771079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204eec7-0ef0-460a-9002-a192588fb721 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.771079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204eec7-0ef0-460a-9002-a192588fb721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.771079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204eec7-0ef0-460a-9002-a192588fb721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.771079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204eec7-0ef0-460a-9002-a192588fb721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.771079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204eec7-0ef0-460a-9002-a192588fb721 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.779614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1145/2318] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:48:48.783930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.795767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694a3be-f164-474d-b29f-5c24cdf3fe9a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:48.801120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:48.802089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.802439 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.802781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] PATH_INFO: `/v3/projects/86cb5d6daebc439293f0f430a6235ba5/users/095fb5105a82489ea5d6485181018270/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.805210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cb5d6daebc439293f0f430a6235ba5, user_id=095fb5105a82489ea5d6485181018270, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:48.808139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:48.822765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1174/2319] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:48:48 2022] PUT /identity/v3/projects/86cb5d6daebc439293f0f430a6235ba5/users/095fb5105a82489ea5d6485181018270/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:48:48.831597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:48.831894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:48.832217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:48.859746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] MFA Rules not processed for user `095fb5105a82489ea5d6485181018270`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:48:48.861031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:48.928479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1146/2320] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:48:48.939884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:48.941615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:49.001228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7151b-d07b-449a-9ee3-db776837a57b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.004870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.005511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.005854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.006073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.015871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.017424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.027686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1175/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:49.277604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:49.290441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60846d-3ebb-4f20-b916-0410b4704eaa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:49.294006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:49.294641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:49.294929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:49.295222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:49.354407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:49.355836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:49.364746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1147/2322] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:53.472537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:53.474067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:53.480573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:53.551315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41e4f9-b765-4246-9e3f-637fcb39a331 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:53.557835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:53.558544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:53.558827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:53.559118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:53.560911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:53.622655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:53.624271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:53.634480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1176/2323] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:56.942728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:56.956079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719121c7-f1db-4556-80dd-698c7fde4c10 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:56.961629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:56.962274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:56.962564 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:56.962849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:56.964622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:57.013559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:48:57.023666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:57.025295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:57.034609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1148/2324] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:48:59.720831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:48:59.732511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ce51-29d1-4dc7-86f1-ae8e2d4ecddc service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:48:59.737998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:48:59.738640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:48:59.738926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:48:59.739203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:48:59.741162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:48:59.753303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:48:59.758306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:48:59.765977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1177/2325] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:48:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:02.233249 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:02.234726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:02.293613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd53f9-1785-46ff-b99c-9859beb972fb service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:02.297383 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:02.298029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:02.298314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:02.298591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:02.300370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:02.363859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:02.365332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da5a59-2dfc-4fd6-862c-18e09a0320f5 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:02.374905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1149/2326] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:03.124111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:03.196251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc21845-64e1-437f-8281-65c61cba51e2 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:03.200890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:03.201779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:03.202068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:03.202349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] PATH_INFO: `/v3/users/d7df1a88e8fa4dabadcd4a85ee962e1c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:03.203677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7df1a88e8fa4dabadcd4a85ee962e1c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:03.206106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:03.349865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:03.450879 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7df1a88e8fa4dabadcd4a85ee962e1c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:03.458071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7df1a88e8fa4dabadcd4a85ee962e1c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:03.541656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7df1a88e8fa4dabadcd4a85ee962e1c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:03.542294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7df1a88e8fa4dabadcd4a85ee962e1c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:03.542919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7df1a88e8fa4dabadcd4a85ee962e1c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:03.561927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1178/2327] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:03 2022] DELETE /identity/v3/users/d7df1a88e8fa4dabadcd4a85ee962e1c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:03.566619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:03.567154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:03.567254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:03.585473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:03.586958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:03.655394 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1150/2328] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:03.666692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:03.668270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:03.727723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc854c-fc31-40f5-a49f-0270b46cb359 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:03.732081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:03.732771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:03.733055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:03.733336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:03.743869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:03.748015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:03.755527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1179/2329] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:04.349940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:04.352585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:04.377401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bd156-d3d5-432e-b671-01f6741a860a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:04.385173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:04.386476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:04.386804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:04.387185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] PATH_INFO: `/v3/projects/03cc0e9fc7944bfdb9bf0c4d8d48bf66` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:04.388537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03cc0e9fc7944bfdb9bf0c4d8d48bf66)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:04.390580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:04.551372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:04.561546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03cc0e9fc7944bfdb9bf0c4d8d48bf66', 'request_id': 'req-d71978c6-d22a-4625-99e2-238fc229e293'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.662076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03cc0e9fc7944bfdb9bf0c4d8d48bf66', 'request_id': 'req-d71978c6-d22a-4625-99e2-238fc229e293'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.668354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03cc0e9fc7944bfdb9bf0c4d8d48bf66', 'request_id': 'req-d71978c6-d22a-4625-99e2-238fc229e293'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.681574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] The token cache is being invalidate because project 03cc0e9fc7944bfdb9bf0c4d8d48bf66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:04.688198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.688198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:04.688198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1151/2330] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:04 2022] DELETE /identity/v3/projects/03cc0e9fc7944bfdb9bf0c4d8d48bf66 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:04.688502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:04.699764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b82d1-1850-41f3-9dae-0172ff828f34 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:04.765647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:04.766526 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:04.766811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:04.767094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] PATH_INFO: `/v3/users/095fb5105a82489ea5d6485181018270` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:04.768459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] RBAC: Authorizing `identity:delete_user(user_id=095fb5105a82489ea5d6485181018270)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:04.770440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:05.017221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095fb5105a82489ea5d6485181018270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.019663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:05.027081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095fb5105a82489ea5d6485181018270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.162462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095fb5105a82489ea5d6485181018270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.163243 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095fb5105a82489ea5d6485181018270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.163924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-07b01567-2654-4aa0-82bf-fca8676020da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095fb5105a82489ea5d6485181018270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:05.192390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1180/2331] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:04 2022] DELETE /identity/v3/users/095fb5105a82489ea5d6485181018270 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:05.770065 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:05.772084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:05.786024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71978c6-d22a-4625-99e2-238fc229e293 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:05.846200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:05.847071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:05.847357 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:05.847659 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] PATH_INFO: `/v3/projects/86cb5d6daebc439293f0f430a6235ba5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:05.848892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86cb5d6daebc439293f0f430a6235ba5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:05.852532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:05.979692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86cb5d6daebc439293f0f430a6235ba5', 'request_id': 'req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:06.056038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86cb5d6daebc439293f0f430a6235ba5', 'request_id': 'req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:06.062106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86cb5d6daebc439293f0f430a6235ba5', 'request_id': 'req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:06.074429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] The token cache is being invalidate because project 86cb5d6daebc439293f0f430a6235ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:06.074717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:06.075354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2090f4dd-795a-4f5f-bb54-b7ec5e520b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:06.076956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1152/2332] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:05 2022] DELETE /identity/v3/projects/86cb5d6daebc439293f0f430a6235ba5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:06.082095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.082576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.082671 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.100352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:06.101659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:06.161819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1181/2333] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:06.170148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.170458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.170765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.189810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:06.191007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:06.199735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1153/2334] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:06.204216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.214910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9d2a9-0e2a-4f8c-bff0-3c77691bb509 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.219765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.221696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.221696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.221696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.221976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:06.222066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.223823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.236748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1182/2335] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:06.242161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.255328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cfcba-0cac-4d95-82de-339e68562780 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.260553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.261781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.262134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.262474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.263235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.265268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.357254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1154/2336] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:06.359423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.370627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e897f-b9a1-432e-b799-47833f636b31 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.429197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.430078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.430375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.430662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.431372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.433438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.524870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1183/2337] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:06.528962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.539855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a01e0b-1c88-4e0d-aee7-60baf3f8207e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.544509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.545207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.545533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.545766 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.546488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.548060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.562586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1155/2338] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:06.566049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.576284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f3766-9280-417e-babe-1a1e99849638 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.581189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cef050-384d-42d0-aab1-968e96744976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.581851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef050-384d-42d0-aab1-968e96744976 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.582139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef050-384d-42d0-aab1-968e96744976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.582423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef050-384d-42d0-aab1-968e96744976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.583151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef050-384d-42d0-aab1-968e96744976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.585435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef050-384d-42d0-aab1-968e96744976 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.595263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1184/2339] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:06.599277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.610270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394854e-0f5b-41f3-830d-70748d1ac4a6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.615192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.615871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.616109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.616392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] PATH_INFO: `/v3/projects/43d2f63f52514e2690ff7a5f6046423a/users/537f3cd4dc8b4e57bf1d8d5a8999182b/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.628049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:06.631113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d2f63f52514e2690ff7a5f6046423a, user_id=537f3cd4dc8b4e57bf1d8d5a8999182b, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.633803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:06.712524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1156/2340] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:06 2022] PUT /identity/v3/projects/43d2f63f52514e2690ff7a5f6046423a/users/537f3cd4dc8b4e57bf1d8d5a8999182b/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:06.723106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc207f25-de30-46a2-af07-071b32770402 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.723402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc207f25-de30-46a2-af07-071b32770402 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.723685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc207f25-de30-46a2-af07-071b32770402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.751354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cc207f25-de30-46a2-af07-071b32770402 None None] MFA Rules not processed for user `537f3cd4dc8b4e57bf1d8d5a8999182b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:06.752632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cc207f25-de30-46a2-af07-071b32770402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:06.775086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:06.820901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1185/2341] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:06.832519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:06.895115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cb6f4-75a4-4121-97df-48626e921523 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:06.900623 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:06.901321 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:06.901607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:06.901886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:06.902586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:06.904507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.004058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1157/2342] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:06 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:07.005202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc207f25-de30-46a2-af07-071b32770402 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.016482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc207f25-de30-46a2-af07-071b32770402 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.073094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.074039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.074847 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.075132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.075839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.077834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.162291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1186/2343] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:07 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:07.166344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.176502 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0907ddd-14e2-49da-a0ea-b2f34b23008f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.181888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.182115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.182452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.182735 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.183462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.185131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.197489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1158/2344] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:07.201398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.212444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9c2c7-6736-4df7-9912-577a5bae1886 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.218006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906aa3f-17b6-465a-828d-87801caa183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.218695 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906aa3f-17b6-465a-828d-87801caa183d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.218963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906aa3f-17b6-465a-828d-87801caa183d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.219237 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906aa3f-17b6-465a-828d-87801caa183d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.220000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906aa3f-17b6-465a-828d-87801caa183d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.221690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906aa3f-17b6-465a-828d-87801caa183d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.233198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1187/2345] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:07.237541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.249367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479f40a-c175-44e2-8af5-3a75f6c07119 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.255012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.256264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.257561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.258264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] PATH_INFO: `/v3/projects/fd3f9039bda547e183b04d41535f14ac/users/f88b33e570e744679918a67dcf5b93e0/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.287164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3f9039bda547e183b04d41535f14ac, user_id=f88b33e570e744679918a67dcf5b93e0, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.290732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.402967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1159/2346] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:07 2022] PUT /identity/v3/projects/fd3f9039bda547e183b04d41535f14ac/users/f88b33e570e744679918a67dcf5b93e0/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:07.414502 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.414923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.415287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.445335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] MFA Rules not processed for user `f88b33e570e744679918a67dcf5b93e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:07.446840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:07.517594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1188/2347] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:07.522942 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.585918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fcd2e-41b4-4b2d-b6a1-219c29b91acc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.590744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.591411 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.591694 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.591971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.592673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.594587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.706791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1160/2348] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:07 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:07.711214 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:07.722065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0780f-9093-44e0-9237-4701e5f8e113 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:07.751768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:07.781166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:07.781851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:07.785577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:07.785577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:07.785577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:07.785577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:07.986801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1189/2349] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:07 2022] POST /identity/v3/users => generated 300 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:07.990698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.001891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3ac63-b5fb-49ee-8da2-d0ae190e13b1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.006346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.007473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.007755 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.008026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.008775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.010343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.020597 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1161/2350] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:08.029642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.041669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4fc20-47ff-4c76-87da-8e107128e91f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.046302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.047263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.047552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.047836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] PATH_INFO: `/v3/projects/edb589d7a33f4cb18edab92e720407f2/users/451446fb93b04f7f96e602238530bd16/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.063385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb589d7a33f4cb18edab92e720407f2, user_id=451446fb93b04f7f96e602238530bd16, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.066209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.145499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1190/2351] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:08 2022] PUT /identity/v3/projects/edb589d7a33f4cb18edab92e720407f2/users/451446fb93b04f7f96e602238530bd16/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:08.150203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.160247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea98ff-9dc1-449d-8409-683cf13ef36e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.172464 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:08.216506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44993aed-ba96-4220-a60a-59d802e24558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.217191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44993aed-ba96-4220-a60a-59d802e24558 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.217474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44993aed-ba96-4220-a60a-59d802e24558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.217751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44993aed-ba96-4220-a60a-59d802e24558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.218470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44993aed-ba96-4220-a60a-59d802e24558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.220036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44993aed-ba96-4220-a60a-59d802e24558 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.233505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1162/2352] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:08.234586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.245589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b89746-952e-4d3a-b23d-ccfee8e758ea admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.250795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.251690 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.255565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.255565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] PATH_INFO: `/v3/projects/edb589d7a33f4cb18edab92e720407f2/users/451446fb93b04f7f96e602238530bd16/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.255565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb589d7a33f4cb18edab92e720407f2, user_id=451446fb93b04f7f96e602238530bd16, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.257179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.268166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1191/2353] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:08 2022] PUT /identity/v3/projects/edb589d7a33f4cb18edab92e720407f2/users/451446fb93b04f7f96e602238530bd16/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:08.279819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.280111 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.280385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.307323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] MFA Rules not processed for user `451446fb93b04f7f96e602238530bd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:08.309256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:08.377346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1163/2354] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:08.388172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:08.389987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:08.452144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a37eb2-900a-4558-a766-f5a3802a1391 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:08.456014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:08.456654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:08.456975 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:08.457251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:08.467657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:08.469294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:08.480056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1192/2355] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:09.895653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:09.898256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:09.910216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f1780-da00-4433-acef-b2a0605478c8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:09.915671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c32b8-6265-47cf-8daa-582517ebc46c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:09.916365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c32b8-6265-47cf-8daa-582517ebc46c service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:09.916709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c32b8-6265-47cf-8daa-582517ebc46c service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:09.917123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c32b8-6265-47cf-8daa-582517ebc46c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:09.975748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c32b8-6265-47cf-8daa-582517ebc46c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:09.977354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c32b8-6265-47cf-8daa-582517ebc46c service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:09.986188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1164/2356] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:15.147949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:15.156042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de76f5-4c13-49f6-bdb4-994399f2c59c service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:15.159858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2082-f876-4e54-9ca7-7a54bbe3714c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:15.160497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2082-f876-4e54-9ca7-7a54bbe3714c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:15.160840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2082-f876-4e54-9ca7-7a54bbe3714c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:15.161120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2082-f876-4e54-9ca7-7a54bbe3714c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:15.220801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c2082-f876-4e54-9ca7-7a54bbe3714c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:15.222246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c2082-f876-4e54-9ca7-7a54bbe3714c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:15.231868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1193/2357] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:18.416801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:18.417370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:18.417474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:18.438334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] MFA Rules not processed for user `46d0d621460b4e5b877ef421d3503340`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:18.438334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:18.467319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:18.491916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1165/2358] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:19.296328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.296845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.296949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.319283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:19.320215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:19.352494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:19.393154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1194/2359] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:49:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:19.594382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:19.596221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:19.684476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3643bf-3db2-4730-a616-1fd654407b30 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:19.690188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1e2f3-68e2-4f55-90ec-e0205c354ec7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:19.690852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e2f3-68e2-4f55-90ec-e0205c354ec7 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:19.691027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e2f3-68e2-4f55-90ec-e0205c354ec7 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:19.691309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e2f3-68e2-4f55-90ec-e0205c354ec7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:19.701009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1e2f3-68e2-4f55-90ec-e0205c354ec7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:19.702271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1e2f3-68e2-4f55-90ec-e0205c354ec7 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:19.711310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1166/2360] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:49:20.467264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:20.543805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afe135-edbb-49d4-9b38-2e3fea1041fe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:20.548777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:20.549668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.550204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.550498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] PATH_INFO: `/v3/users/537f3cd4dc8b4e57bf1d8d5a8999182b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.551712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=537f3cd4dc8b4e57bf1d8d5a8999182b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:20.553974 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:20.783999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537f3cd4dc8b4e57bf1d8d5a8999182b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.795472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537f3cd4dc8b4e57bf1d8d5a8999182b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.891294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537f3cd4dc8b4e57bf1d8d5a8999182b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.892215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537f3cd4dc8b4e57bf1d8d5a8999182b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.893232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537f3cd4dc8b4e57bf1d8d5a8999182b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:20.915438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1195/2361] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:20 2022] DELETE /identity/v3/users/537f3cd4dc8b4e57bf1d8d5a8999182b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:20.920140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:20.920638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:20.920753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:20.940621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:20.942824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:21.037106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1167/2362] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:21.048955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:21.050440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:21.118982 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:21.124058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe996f2-1fe4-49c9-887c-e895baf402d2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:21.127798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:21.128559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:21.131374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:21.131374 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:21.149080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:21.150240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:21.162993 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1196/2363] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:21.926189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:21.927978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:21.948572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5c3bd-a850-4394-9eb0-a1581bd4ab6c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:21.957709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:21.959525 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:21.960627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:21.961333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] PATH_INFO: `/v3/projects/43d2f63f52514e2690ff7a5f6046423a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:21.962877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d2f63f52514e2690ff7a5f6046423a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:21.967307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:22.145737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d2f63f52514e2690ff7a5f6046423a', 'request_id': 'req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.213757 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d2f63f52514e2690ff7a5f6046423a', 'request_id': 'req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.229637 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d2f63f52514e2690ff7a5f6046423a', 'request_id': 'req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.232140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:22.260939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] The token cache is being invalidate because project 43d2f63f52514e2690ff7a5f6046423a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:22.261451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.262294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.268162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1168/2364] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:21 2022] DELETE /identity/v3/projects/43d2f63f52514e2690ff7a5f6046423a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:22.272011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:22.283781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea04ec-5ef4-4e37-8f8f-671f4ee5669e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:22.378963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:22.379898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:22.380188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:22.380468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] PATH_INFO: `/v3/users/f88b33e570e744679918a67dcf5b93e0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:22.382213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88b33e570e744679918a67dcf5b93e0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:22.386991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:22.656210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88b33e570e744679918a67dcf5b93e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.668716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88b33e570e744679918a67dcf5b93e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.793868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88b33e570e744679918a67dcf5b93e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.805941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88b33e570e744679918a67dcf5b93e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.805941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88b33e570e744679918a67dcf5b93e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:22.820814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1197/2365] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:22 2022] DELETE /identity/v3/users/f88b33e570e744679918a67dcf5b93e0 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:23.924397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:23.928559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:23.949982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c989b2-5ae1-44f5-a2ec-2f4b45bc3b47 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.048776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.049841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.050140 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.050560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] PATH_INFO: `/v3/projects/fd3f9039bda547e183b04d41535f14ac` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.051836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3f9039bda547e183b04d41535f14ac)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.055198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.236716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3f9039bda547e183b04d41535f14ac', 'request_id': 'req-fd9203a9-2315-4bc6-b112-8b9e03587ae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.346820 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3f9039bda547e183b04d41535f14ac', 'request_id': 'req-fd9203a9-2315-4bc6-b112-8b9e03587ae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.355381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3f9039bda547e183b04d41535f14ac', 'request_id': 'req-fd9203a9-2315-4bc6-b112-8b9e03587ae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.374397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] The token cache is being invalidate because project fd3f9039bda547e183b04d41535f14ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:24.374726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.375888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.379160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1169/2366] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:23 2022] DELETE /identity/v3/projects/fd3f9039bda547e183b04d41535f14ac => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:24.384486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:24.386148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:24.423196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d85c5-22ba-4306-be03-66d300f514fe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:24.572410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:24.573362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:24.573655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:24.574221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] PATH_INFO: `/v3/users/451446fb93b04f7f96e602238530bd16` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:24.575358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451446fb93b04f7f96e602238530bd16)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:24.577383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:24.593484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:24.788611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451446fb93b04f7f96e602238530bd16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.803283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451446fb93b04f7f96e602238530bd16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.899403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451446fb93b04f7f96e602238530bd16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.901973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451446fb93b04f7f96e602238530bd16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.902770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8d678c-b755-4975-9f1a-9261ff716829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451446fb93b04f7f96e602238530bd16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:24.969144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1198/2367] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:24 2022] DELETE /identity/v3/users/451446fb93b04f7f96e602238530bd16 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:25.687377 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:25.690401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:25.714166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9203a9-2315-4bc6-b112-8b9e03587ae0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:25.791495 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:25.792406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:25.792718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:25.793043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] PATH_INFO: `/v3/projects/edb589d7a33f4cb18edab92e720407f2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:25.795816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb589d7a33f4cb18edab92e720407f2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:25.797334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:25.907926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:25.928157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb589d7a33f4cb18edab92e720407f2', 'request_id': 'req-03eb120d-8334-41dd-8ecb-da48eaabe019'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:26.032794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb589d7a33f4cb18edab92e720407f2', 'request_id': 'req-03eb120d-8334-41dd-8ecb-da48eaabe019'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:26.040987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb589d7a33f4cb18edab92e720407f2', 'request_id': 'req-03eb120d-8334-41dd-8ecb-da48eaabe019'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:26.054702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] The token cache is being invalidate because project edb589d7a33f4cb18edab92e720407f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:26.054996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:26.055695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03eb120d-8334-41dd-8ecb-da48eaabe019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:26.061784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1170/2368] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:25 2022] DELETE /identity/v3/projects/edb589d7a33f4cb18edab92e720407f2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:26.063599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.064079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.064177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.086300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:26.087996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:26.168392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1199/2369] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:26.176899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.177282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.178361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.199992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:26.201279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:26.215918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1171/2370] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:26.217448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.228769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf9f5f-da5c-4978-9b2d-761ee2c97836 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.234814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.235582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.236108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.236589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.237432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:26.237814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.239507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.253721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1200/2371] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:26.258302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.272858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c2171-7c89-4370-880e-94ec677bb62d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.278276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.279098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.279473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.279844 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.280704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.283146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.385455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1172/2372] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:26 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:26.390909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.418991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b71d15-b890-43cc-ba85-d83afe47877f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.491007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.491870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.492163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.492449 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.495831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.495831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.580800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1201/2373] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:26 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:26.585000 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.608803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d28433-8e6f-4c7e-9399-b2ee8ffe8577 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.611186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.611888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.616312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.616312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.616312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.616312 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.632305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1173/2374] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:26.636846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.653115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f583394-09a7-46eb-bed1-0f52b8694cc4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.658219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a982fa-11da-4130-9bf7-b6d6317a55e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.658926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a982fa-11da-4130-9bf7-b6d6317a55e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.659219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a982fa-11da-4130-9bf7-b6d6317a55e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.659504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a982fa-11da-4130-9bf7-b6d6317a55e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.660271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a982fa-11da-4130-9bf7-b6d6317a55e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.662239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a982fa-11da-4130-9bf7-b6d6317a55e7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:26.664006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:26.676420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1202/2375] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:26.681192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:26.692439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566b04f-f7b7-4b3f-880d-bbb99de2fd50 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:26.698702 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:26.699725 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:26.700088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:26.700373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] PATH_INFO: `/v3/projects/e7c74aee672a48da9616a18d44909be4/users/c58fc284e7824532bf5252a0502926ff/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:26.717450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c74aee672a48da9616a18d44909be4, user_id=c58fc284e7824532bf5252a0502926ff, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:26.720324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.090770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1174/2376] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:26 2022] PUT /identity/v3/projects/e7c74aee672a48da9616a18d44909be4/users/c58fc284e7824532bf5252a0502926ff/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:27.102881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.107775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.107775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.131983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] MFA Rules not processed for user `c58fc284e7824532bf5252a0502926ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:27.133221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:27.202793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1203/2377] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:27.207629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:27.271359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb36926-1af6-4919-9d7c-2885a5a81a4e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:27.276098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:27.276798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.279708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.279708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.279708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.280001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.425591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1175/2378] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:27 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:27.430326 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:27.443684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8ff16-131d-4d51-a13b-fc32d0bdb2c9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:27.517589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:27.518387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.518574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.518875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.520479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.522916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.631473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1204/2379] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:27 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:27.632860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:27.650971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04aa5-aae9-4aeb-a767-a8538beb41f6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:27.663950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:27.664672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.665010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.665270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.666014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.667651 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.689017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1176/2380] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:27.693121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:27.705100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2193-4fbc-4647-92b7-02f86cea8468 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:27.710412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:27.711557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.711966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.712353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] PATH_INFO: `/v3/projects/2440428f38b44e7c983fcb26e30058e2/users/e8dc44b75fbe471992ff5215fe8e396c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.743253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2440428f38b44e7c983fcb26e30058e2, user_id=e8dc44b75fbe471992ff5215fe8e396c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.746161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.848655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1205/2381] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:27 2022] PUT /identity/v3/projects/2440428f38b44e7c983fcb26e30058e2/users/e8dc44b75fbe471992ff5215fe8e396c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:27.852452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:27.864387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5d5c5-f1ee-4cad-bfb4-a006f3163f48 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:27.887715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:27.932570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf307b57-91ca-4ef6-b5e9-63498d3d3d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:27.933517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf307b57-91ca-4ef6-b5e9-63498d3d3d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.933862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf307b57-91ca-4ef6-b5e9-63498d3d3d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.934166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf307b57-91ca-4ef6-b5e9-63498d3d3d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.934909 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf307b57-91ca-4ef6-b5e9-63498d3d3d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.936526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf307b57-91ca-4ef6-b5e9-63498d3d3d1a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.952269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1177/2382] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:27.956435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:27.968096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ec688-f097-4839-863a-e999636bbd71 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:27.974431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:27.975267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:27.975580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:27.975958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] PATH_INFO: `/v3/projects/2440428f38b44e7c983fcb26e30058e2/users/e8dc44b75fbe471992ff5215fe8e396c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:27.978200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2440428f38b44e7c983fcb26e30058e2, user_id=e8dc44b75fbe471992ff5215fe8e396c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:27.980758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:27.993189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1206/2383] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:27 2022] PUT /identity/v3/projects/2440428f38b44e7c983fcb26e30058e2/users/e8dc44b75fbe471992ff5215fe8e396c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:28.003822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:28.004595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:28.005329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:28.048641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] MFA Rules not processed for user `e8dc44b75fbe471992ff5215fe8e396c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:28.050756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:28.135436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1178/2384] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:28.146503 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:28.148197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:28.197007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:28.213472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9caba23-ed60-4eb0-a1aa-a759015449c2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:28.218231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:28.218896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:28.219200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:28.219514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:28.232627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:28.234110 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:28.245189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1207/2385] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:28.375306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:28.386348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4554dd-8aec-4d14-a1d7-17fecc1e52c5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:28.390734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d222254-7e86-4f18-9561-9c940fa92114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:28.391478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d222254-7e86-4f18-9561-9c940fa92114 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:28.391835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d222254-7e86-4f18-9561-9c940fa92114 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:28.392120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d222254-7e86-4f18-9561-9c940fa92114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:28.461112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d222254-7e86-4f18-9561-9c940fa92114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:28.462558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d222254-7e86-4f18-9561-9c940fa92114 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:28.473648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1179/2386] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:31.869242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:31.933225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcad32-4cb0-4f03-a4d3-2b22145ff787 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:31.937839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:31.938715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:31.939012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:31.939293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] PATH_INFO: `/v3/users/c58fc284e7824532bf5252a0502926ff` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:31.940483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58fc284e7824532bf5252a0502926ff)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:31.942451 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:32.268806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58fc284e7824532bf5252a0502926ff'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:32.278893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58fc284e7824532bf5252a0502926ff'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:32.421572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58fc284e7824532bf5252a0502926ff'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:32.422271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58fc284e7824532bf5252a0502926ff'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:32.423197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58fc284e7824532bf5252a0502926ff'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:32.450733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1208/2387] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:31 2022] DELETE /identity/v3/users/c58fc284e7824532bf5252a0502926ff => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:32.456935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:32.458277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:32.459396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:32.501515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:32.502807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:32.551488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:32.570908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1180/2388] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:32.589952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:32.591796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:32.668344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07912b-d2d5-40d0-be53-d3ffaee33218 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:32.672605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:32.673460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:32.673753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:32.674030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:32.691786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:32.693715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:32.704585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1209/2389] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:33.175661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:33.175661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:33.185856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24342867-6d22-4036-aef5-6c3a1d16e7a8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:33.192367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:33.193401 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:33.193695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:33.193980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] PATH_INFO: `/v3/projects/e7c74aee672a48da9616a18d44909be4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:33.195421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c74aee672a48da9616a18d44909be4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:33.197381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:33.376572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c74aee672a48da9616a18d44909be4', 'request_id': 'req-176bdc10-3ca0-498a-ae68-de5708422acf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.461038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c74aee672a48da9616a18d44909be4', 'request_id': 'req-176bdc10-3ca0-498a-ae68-de5708422acf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.467132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c74aee672a48da9616a18d44909be4', 'request_id': 'req-176bdc10-3ca0-498a-ae68-de5708422acf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.478727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] The token cache is being invalidate because project e7c74aee672a48da9616a18d44909be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:33.479017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.479656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.481244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1181/2390] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:33 2022] DELETE /identity/v3/projects/e7c74aee672a48da9616a18d44909be4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:33.485121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:33.490587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:33.495907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60366b29-74ca-4641-b67d-6c72fcfe470b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:33.553775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:33.554655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:33.554945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:33.555234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] PATH_INFO: `/v3/users/e8dc44b75fbe471992ff5215fe8e396c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:33.556458 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8dc44b75fbe471992ff5215fe8e396c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:33.558433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:33.861790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8dc44b75fbe471992ff5215fe8e396c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.872457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8dc44b75fbe471992ff5215fe8e396c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.970172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8dc44b75fbe471992ff5215fe8e396c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.971185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8dc44b75fbe471992ff5215fe8e396c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:33.971876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcfffcfb-7c09-43e2-b09f-70cf05eda9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8dc44b75fbe471992ff5215fe8e396c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:34.001345 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1210/2391] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:33 2022] DELETE /identity/v3/users/e8dc44b75fbe471992ff5215fe8e396c => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:34.468754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.471913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:34.486831 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bdc10-3ca0-498a-ae68-de5708422acf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:34.546582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:34.547452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.547739 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.548021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] PATH_INFO: `/v3/projects/2440428f38b44e7c983fcb26e30058e2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.549298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2440428f38b44e7c983fcb26e30058e2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.551314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.682056 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2440428f38b44e7c983fcb26e30058e2', 'request_id': 'req-aa450e25-3a3b-4480-88f5-ac44d3ca816c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:34.787176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2440428f38b44e7c983fcb26e30058e2', 'request_id': 'req-aa450e25-3a3b-4480-88f5-ac44d3ca816c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:34.793392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2440428f38b44e7c983fcb26e30058e2', 'request_id': 'req-aa450e25-3a3b-4480-88f5-ac44d3ca816c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:34.801001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:34.806674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] The token cache is being invalidate because project 2440428f38b44e7c983fcb26e30058e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:34.806963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:34.808995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa450e25-3a3b-4480-88f5-ac44d3ca816c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:34.809305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1182/2392] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:34 2022] DELETE /identity/v3/projects/2440428f38b44e7c983fcb26e30058e2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:34.814152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.814613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.814747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.834355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:34.835619 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:34.899730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1211/2393] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:34.916501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.916833 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.917115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.935762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:34.937016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:34.945618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1183/2394] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:34.950420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.969138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530fa7-5736-4b50-8afe-a3e3fef26a6c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:34.969138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:34.969138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:34.969138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:34.969499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:34.969499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:34.969499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:34.969760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:34.982001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1212/2395] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:34.986641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:34.997669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3025f5-cd62-4fd4-aa4d-139e263afefc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:35.003128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:35.003866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:35.004223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:35.004798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:35.005592 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:35.009246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:35.352370 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1184/2396] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:34 2022] POST /identity/v3/projects => generated 387 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:35.353752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:35.363848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aa07e-cf4e-4557-9384-25a3d7c46f57 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:35.433251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:35.433251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:35.433251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:35.433251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:35.434060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:35.434060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:35.553765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1213/2397] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:35 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:35.558076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:35.572324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5a37-a3fd-4ec4-b5ba-cb164e3c7546 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:35.577878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:35.578929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:35.579322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:35.579796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:35.580921 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:35.582684 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:35.595462 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1185/2398] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:35.600649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:35.611134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac40e8-83a0-4413-a5d5-1b9aacfe2873 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:35.617270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfc772-26cc-4d2e-a939-3b7a6290bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:35.617926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfc772-26cc-4d2e-a939-3b7a6290bb93 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:35.618211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfc772-26cc-4d2e-a939-3b7a6290bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:35.618490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfc772-26cc-4d2e-a939-3b7a6290bb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:35.619216 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfc772-26cc-4d2e-a939-3b7a6290bb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:35.620923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfc772-26cc-4d2e-a939-3b7a6290bb93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:35.631505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1214/2399] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:35.635835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:35.646985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54cef-3038-42d1-9e5d-cfb6e24810e2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:35.651606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:35.652491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:35.652812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:35.653097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] PATH_INFO: `/v3/projects/fe3f20cc26a9468ab8e15e39ddb2d023/users/d955ca5cc8ea4719a14bc07bc030eae0/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:35.668352 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3f20cc26a9468ab8e15e39ddb2d023, user_id=d955ca5cc8ea4719a14bc07bc030eae0, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:35.678008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:35.776595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:35.865869 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1186/2400] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:35 2022] PUT /identity/v3/projects/fe3f20cc26a9468ab8e15e39ddb2d023/users/d955ca5cc8ea4719a14bc07bc030eae0/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:35.877840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:35.877840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:35.878301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:35.915031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] MFA Rules not processed for user `d955ca5cc8ea4719a14bc07bc030eae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:35.916435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:35.985167 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1215/2401] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:35.990123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.053756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561340-373e-44af-adea-8cda31ba88d6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.060046 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.061196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.061196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.061398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.062076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.063964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.191826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1187/2402] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:35 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:49:36.193311 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.204840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5e6e3-0d3d-4f63-ac81-ce3fdc361bad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.283410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.284212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.284579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.284933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.285660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.289218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.408190 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1216/2403] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:36 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:36.409595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.421412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bf257-235a-498c-846c-e1db144b1755 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.426229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.426915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.427279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.427653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.428604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.430294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.445966 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1188/2404] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:36.447022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.457925 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cffdf-9ede-4ba0-991d-0e6a2c43cd2a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.463360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.464265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.464556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.464874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] PATH_INFO: `/v3/projects/8f02797f13be42dc82d50876dfabc5cc/users/3c935671275c459eafc5fcaabac17d0e/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.481529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f02797f13be42dc82d50876dfabc5cc, user_id=3c935671275c459eafc5fcaabac17d0e, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.484446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.559109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1217/2405] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:36 2022] PUT /identity/v3/projects/8f02797f13be42dc82d50876dfabc5cc/users/3c935671275c459eafc5fcaabac17d0e/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:49:36.563779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.575026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690182-3629-43c7-92a6-ef5e177f16e6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.647733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93baffdf-7635-41c8-a2e4-2c12289c8586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.648752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baffdf-7635-41c8-a2e4-2c12289c8586 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.649066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baffdf-7635-41c8-a2e4-2c12289c8586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.649358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baffdf-7635-41c8-a2e4-2c12289c8586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.650164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93baffdf-7635-41c8-a2e4-2c12289c8586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.651783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93baffdf-7635-41c8-a2e4-2c12289c8586 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.666917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1189/2406] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:36.671538 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.684412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7520d3-f009-4329-96a1-83e8656aec46 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.689853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.690871 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.691161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.691444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] PATH_INFO: `/v3/projects/8f02797f13be42dc82d50876dfabc5cc/users/3c935671275c459eafc5fcaabac17d0e/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.694144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f02797f13be42dc82d50876dfabc5cc, user_id=3c935671275c459eafc5fcaabac17d0e, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.697641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.718877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1218/2407] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:36 2022] PUT /identity/v3/projects/8f02797f13be42dc82d50876dfabc5cc/users/3c935671275c459eafc5fcaabac17d0e/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:36.730134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.730435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.730720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.739302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:36.765120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] MFA Rules not processed for user `3c935671275c459eafc5fcaabac17d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:36.769186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:36.830806 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:36.852974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1190/2408] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:36.859976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:36.861782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:36.940183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c4645-0e97-4c07-9928-a5246a1c551e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:36.944208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f49c-9a35-4e96-bf94-ce387a4c4fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:36.950002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f49c-9a35-4e96-bf94-ce387a4c4fde service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:36.950002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f49c-9a35-4e96-bf94-ce387a4c4fde service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:36.950002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f49c-9a35-4e96-bf94-ce387a4c4fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:36.956335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7f49c-9a35-4e96-bf94-ce387a4c4fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:36.958081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7f49c-9a35-4e96-bf94-ce387a4c4fde service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:36.978769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1219/2409] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:38.323543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:38.386828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee108db-d5ca-4cd5-9d7a-0bb1528fe2c5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:38.391593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:38.392500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:38.392781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:38.393061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] PATH_INFO: `/v3/users/d955ca5cc8ea4719a14bc07bc030eae0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:38.394274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d955ca5cc8ea4719a14bc07bc030eae0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:38.396962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:38.647431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd955ca5cc8ea4719a14bc07bc030eae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:38.659339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd955ca5cc8ea4719a14bc07bc030eae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:38.666224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd955ca5cc8ea4719a14bc07bc030eae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:38.752540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd955ca5cc8ea4719a14bc07bc030eae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:38.753197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd955ca5cc8ea4719a14bc07bc030eae0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:38.760230 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1191/2410] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:38 2022] DELETE /identity/v3/users/d955ca5cc8ea4719a14bc07bc030eae0 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:38.764768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:38.765232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:38.765330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:38.783629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:38.784790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:38.845685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1220/2411] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:38.855231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:38.856705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:38.869223 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:38.915670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde5401-5a96-46bd-ab3f-304d510eb73b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:38.919281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d1ebb-66e6-44dc-b525-6c98b05a2b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:38.919928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d1ebb-66e6-44dc-b525-6c98b05a2b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:38.920211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d1ebb-66e6-44dc-b525-6c98b05a2b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:38.920487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d1ebb-66e6-44dc-b525-6c98b05a2b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:38.930258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d1ebb-66e6-44dc-b525-6c98b05a2b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:38.931699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d1ebb-66e6-44dc-b525-6c98b05a2b65 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:38.941675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1192/2412] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:39.443480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:39.444932 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:39.459322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4173-3cbb-419e-8429-ae1041dd07a3 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:39.459828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4173-3cbb-419e-8429-ae1041dd07a3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:39.460091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4173-3cbb-419e-8429-ae1041dd07a3 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:39.461752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1193/2413] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:49:39.470589 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4cc0c-1351-4f79-906b-ecbaa385b051 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:39.472699 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:39.473011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:39.473295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:39.477984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:39.480821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:39.480821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:39.480821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] PATH_INFO: `/v3/projects/fe3f20cc26a9468ab8e15e39ddb2d023` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:39.480821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3f20cc26a9468ab8e15e39ddb2d023)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:39.483075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:39.504515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:39.506076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:39.573552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1194/2414] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:49:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:39.586436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:39.587795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:39.627624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3f20cc26a9468ab8e15e39ddb2d023', 'request_id': 'req-5f5d0d78-178b-4803-823b-db59bc974dbe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:39.634389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3f20cc26a9468ab8e15e39ddb2d023', 'request_id': 'req-5f5d0d78-178b-4803-823b-db59bc974dbe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:39.646617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5fe18-61fb-4708-9ec5-2d920aebf382 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:39.695441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505f80e-24eb-4bef-84be-77042232b123 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:39.696087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505f80e-24eb-4bef-84be-77042232b123 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:39.696373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505f80e-24eb-4bef-84be-77042232b123 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:39.696654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505f80e-24eb-4bef-84be-77042232b123 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:39.709192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:39.741106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3f20cc26a9468ab8e15e39ddb2d023', 'request_id': 'req-5f5d0d78-178b-4803-823b-db59bc974dbe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:39.746945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:39.753556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] The token cache is being invalidate because project fe3f20cc26a9468ab8e15e39ddb2d023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:39.753846 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:39.754476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:39.756152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1221/2415] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:39 2022] DELETE /identity/v3/projects/fe3f20cc26a9468ab8e15e39ddb2d023 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:39.759991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:39.761323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:39.765263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505f80e-24eb-4bef-84be-77042232b123 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:39.766700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505f80e-24eb-4bef-84be-77042232b123 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:39.778496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1195/2416] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:39.827744 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d0d78-178b-4803-823b-db59bc974dbe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:39.834292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:39.835274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:39.835563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:39.835844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] PATH_INFO: `/v3/users/3c935671275c459eafc5fcaabac17d0e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:39.837127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c935671275c459eafc5fcaabac17d0e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:39.839039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:39.994732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165321b-e03f-4f43-9e6d-8e8497da405a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:39.995023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165321b-e03f-4f43-9e6d-8e8497da405a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:39.995303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165321b-e03f-4f43-9e6d-8e8497da405a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:39.996753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1196/2417] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:49:40.001038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:40.001325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:40.004278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:40.019961 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] MFA Rules not processed for user `ce0633dadc8b43c5a13f52cace5bb85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:40.021203 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:40.046639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c935671275c459eafc5fcaabac17d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:40.054713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c935671275c459eafc5fcaabac17d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:40.073895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1197/2418] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:49:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:40.086590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:40.087952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:40.098271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b50dd-32ad-4b84-bc01-110daf41b34c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:40.102260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c98231-76f8-473d-8aea-4f7fc9132b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:40.102994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c98231-76f8-473d-8aea-4f7fc9132b62 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:40.103278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c98231-76f8-473d-8aea-4f7fc9132b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:40.103556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c98231-76f8-473d-8aea-4f7fc9132b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:40.110773 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c98231-76f8-473d-8aea-4f7fc9132b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:40.113175 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c98231-76f8-473d-8aea-4f7fc9132b62 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:40.119944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1198/2419] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:49:40.133234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c935671275c459eafc5fcaabac17d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:40.133866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c935671275c459eafc5fcaabac17d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:40.134477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b80bca5-9e74-4427-883e-addf498a0240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c935671275c459eafc5fcaabac17d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:40.155247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1222/2420] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:49:39 2022] DELETE /identity/v3/users/3c935671275c459eafc5fcaabac17d0e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:49:40.205830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc44ce-6f18-4cee-9132-447401967094 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:40.206183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc44ce-6f18-4cee-9132-447401967094 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:40.206477 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc44ce-6f18-4cee-9132-447401967094 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:40.209085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1199/2421] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:49:40.214650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:40.214940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:40.215224 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:40.234801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] MFA Rules not processed for user `aa1f5d0bba8a4178977c19b7503ef6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:40.236238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:40.320307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1223/2422] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:49:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:40.331750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc44ce-6f18-4cee-9132-447401967094 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:40.333865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-01fc44ce-6f18-4cee-9132-447401967094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:40.394566 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc44ce-6f18-4cee-9132-447401967094 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:40.399339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7bf67-0e98-45a3-90a1-a266fa25d95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:40.400041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7bf67-0e98-45a3-90a1-a266fa25d95a service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:40.400329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7bf67-0e98-45a3-90a1-a266fa25d95a service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:40.400607 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7bf67-0e98-45a3-90a1-a266fa25d95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:40.415388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7bf67-0e98-45a3-90a1-a266fa25d95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:40.417749 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7bf67-0e98-45a3-90a1-a266fa25d95a service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:40.429433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1200/2423] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:40.634147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:40.635612 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:40.647400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:40.704917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746b50b-7222-4ad5-84c9-3765cb6818d1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:40.710498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:40.711365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:40.711653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:40.711934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] PATH_INFO: `/v3/projects/8f02797f13be42dc82d50876dfabc5cc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:40.713169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f02797f13be42dc82d50876dfabc5cc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:40.715146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:40.876803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f02797f13be42dc82d50876dfabc5cc', 'request_id': 'req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:40.882874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f02797f13be42dc82d50876dfabc5cc', 'request_id': 'req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:41.015447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f02797f13be42dc82d50876dfabc5cc', 'request_id': 'req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:41.028621 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] The token cache is being invalidate because project 8f02797f13be42dc82d50876dfabc5cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:49:41.028944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:41.029587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fd1cbe3c-4bb5-49f1-92e3-d37d90b993af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:49:41.031130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1224/2424] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:49:40 2022] DELETE /identity/v3/projects/8f02797f13be42dc82d50876dfabc5cc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:41.036796 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.037319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.037625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.055824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:41.057091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:41.092841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:41.118526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1201/2425] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:41.126460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020125-af7e-476f-b518-349ea0453725 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.126754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020125-af7e-476f-b518-349ea0453725 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.127084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020125-af7e-476f-b518-349ea0453725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.145408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2d020125-af7e-476f-b518-349ea0453725 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:41.147135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2d020125-af7e-476f-b518-349ea0453725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:41.155177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1225/2426] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:41.159253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.169674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1666-a779-4d2c-98b7-8a2d02928f9b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.174225 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.174871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.175157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.175433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.176158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:49:41.176470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.178374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:41.195579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1202/2427] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:49:41.203184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d020125-af7e-476f-b518-349ea0453725 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.214497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d020125-af7e-476f-b518-349ea0453725 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.219388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.220103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.220365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.220661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.221511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.223408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:41.344737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1226/2428] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:41 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:41.349252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.359344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f9e4f-ac16-494c-bfb1-d003cb0435b0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.420994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.421586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.421872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.422149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.422940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.424894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:41.546810 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1203/2429] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:41 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:41.553149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.563396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af988167-216a-42fc-b92b-e1e7dfa7df13 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.567956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.568610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.568924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.569201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.569924 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.571535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:41.582091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1227/2430] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:41.586395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.596987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76607-8ccf-4d98-8f53-9ac787e2f75a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.602196 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3885c-6a5d-43ed-a5c6-7a41ddc286e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.602854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3885c-6a5d-43ed-a5c6-7a41ddc286e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.603170 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3885c-6a5d-43ed-a5c6-7a41ddc286e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.603454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3885c-6a5d-43ed-a5c6-7a41ddc286e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.604188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3885c-6a5d-43ed-a5c6-7a41ddc286e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.606002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3885c-6a5d-43ed-a5c6-7a41ddc286e2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:41.617097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1204/2431] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:41.621445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.631703 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239271d-ca58-4d84-a759-d03e59edbe37 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:41.636350 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:41.637275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.637561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.637845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] PATH_INFO: `/v3/projects/c6040be3467648939b7ff7138c05482c/users/80ff0f55699841c38e075e7709c30553/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.652566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6040be3467648939b7ff7138c05482c, user_id=80ff0f55699841c38e075e7709c30553, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:41.655269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:41.857455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1228/2432] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:41 2022] PUT /identity/v3/projects/c6040be3467648939b7ff7138c05482c/users/80ff0f55699841c38e075e7709c30553/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:41.868728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cb942-e092-4849-b2cd-f8f436053ec6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:41.869032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cb942-e092-4849-b2cd-f8f436053ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:41.869323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cb942-e092-4849-b2cd-f8f436053ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:41.889057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-496cb942-e092-4849-b2cd-f8f436053ec6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:41.890274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-496cb942-e092-4849-b2cd-f8f436053ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:41.961433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1205/2433] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:41.971119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:41.973087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:42.009863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:42.031507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b420ea1-075b-424e-ab3f-b52388e47700 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:42.035219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:42.035900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:42.036316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:42.036729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:42.045723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:42.047150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:42.057196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1229/2434] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:43.504038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3a636-433f-428f-b6a7-d8ca7f1d7e09 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:43.504490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3a636-433f-428f-b6a7-d8ca7f1d7e09 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:43.504490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3a636-433f-428f-b6a7-d8ca7f1d7e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:43.525428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b6b3a636-433f-428f-b6a7-d8ca7f1d7e09 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:43.527645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b6b3a636-433f-428f-b6a7-d8ca7f1d7e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:43.576841 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:43.598703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1206/2435] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:49:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:43.612629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:43.614573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:43.663342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:43.663625 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:43.663904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:43.696701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] MFA Rules not processed for user `80ff0f55699841c38e075e7709c30553`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:43.698155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:43.709533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cc6a2-e16a-4728-a2ec-24d1fb37adcf service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:43.715696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:43.716834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:43.717202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:43.717587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:43.733802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:43.733802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:43.741712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1230/2436] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:49:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:49:43.774976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1207/2437] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:49:43.779400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:43.790618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bf786-8f80-41df-9a1a-5b0dae64742a service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:43.795562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:43.796233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:43.796519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:43.796913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:43.797657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:43.799697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:44.191151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1231/2438] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:49:43 2022] POST /identity/v3/projects => generated 373 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:44.193019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:44.206928 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32478195-ad9e-44c2-9714-f8b449b68d2a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:44.339530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:44.340219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:44.340510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:44.340839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:44.341603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:44.343678 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:44.532410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1208/2439] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:49:44 2022] POST /identity/v3/users => generated 293 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:49:44.534486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:44.550964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db63e1-f3da-4f9c-a45a-6418bc5af1c2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:44.558287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:44.559129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:44.559525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:44.559913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:44.560962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:44.562834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:44.582080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1232/2440] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:44.586030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:44.597548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d180-c02a-41aa-b1df-bb99f72741ad admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:44.603695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:44.605454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:44.605454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:44.605454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] PATH_INFO: `/v3/projects/e85c29fed4c24951b047954a292a418a/users/7215a95a5f0840d7921e9f601f0309d9/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:44.624894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85c29fed4c24951b047954a292a418a, user_id=7215a95a5f0840d7921e9f601f0309d9, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:44.627743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:44.749614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1209/2441] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:44 2022] PUT /identity/v3/projects/e85c29fed4c24951b047954a292a418a/users/7215a95a5f0840d7921e9f601f0309d9/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:44.753988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:44.768191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c27f2-1b89-45be-9265-84f16ac19665 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:44.838298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85433d-2210-454e-b1a7-a6d8597190f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:44.839024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85433d-2210-454e-b1a7-a6d8597190f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:44.841475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85433d-2210-454e-b1a7-a6d8597190f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:44.841475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85433d-2210-454e-b1a7-a6d8597190f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:44.841475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85433d-2210-454e-b1a7-a6d8597190f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:44.842006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85433d-2210-454e-b1a7-a6d8597190f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:44.861181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:44.861607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1233/2442] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:49:44.872258 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efefd4-b594-4737-9c28-1530c8979f2f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:44.877215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:44.878113 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:44.878429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:44.878668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] PATH_INFO: `/v3/projects/e85c29fed4c24951b047954a292a418a/users/7215a95a5f0840d7921e9f601f0309d9/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:44.881035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85c29fed4c24951b047954a292a418a, user_id=7215a95a5f0840d7921e9f601f0309d9, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:44.883792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:44.896545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1210/2443] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:49:44 2022] PUT /identity/v3/projects/e85c29fed4c24951b047954a292a418a/users/7215a95a5f0840d7921e9f601f0309d9/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:49:46.721603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.722852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.723291 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:46.781334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] MFA Rules not processed for user `7215a95a5f0840d7921e9f601f0309d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:46.782923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:46.804833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:46.904459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1234/2444] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:49:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:46.917351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:46.919207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:46.959713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:46.984035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99443-41d1-4d40-8bdb-dc5cd3d17070 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:46.987708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444f91-62b4-4c01-bdfb-49b77efc5835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:46.988417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444f91-62b4-4c01-bdfb-49b77efc5835 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:46.988719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444f91-62b4-4c01-bdfb-49b77efc5835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:46.989036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444f91-62b4-4c01-bdfb-49b77efc5835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:47.060986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77444f91-62b4-4c01-bdfb-49b77efc5835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:47.062320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77444f91-62b4-4c01-bdfb-49b77efc5835 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:47.071990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1211/2445] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:48.999161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:49.000579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:49.013768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab066-b5b0-4752-9662-1629d4b82f36 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:49.017523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:49.020890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:49.020890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:49.020890 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:49.029033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:49.030672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:49.042738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1235/2446] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:49:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:49:57.067337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=33 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:59.038074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=27 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:49:59.688906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87a65-b6ea-4595-a704-d5192f40f31f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:59.689444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87a65-b6ea-4595-a704-d5192f40f31f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:59.689444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87a65-b6ea-4595-a704-d5192f40f31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:59.709308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-09d87a65-b6ea-4595-a704-d5192f40f31f None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:49:59.710530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-09d87a65-b6ea-4595-a704-d5192f40f31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:49:59.767745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1212/2447] 10.222.0.47 () {58 vars in 967 bytes} [Mon Jul 18 10:49:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:49:59.781068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:49:59.852262 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6cf0d-a3a9-48f1-826a-6b0e7a8b2e4e service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:49:59.857419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad508cd-273c-4ba3-9682-3c8ca30bf271 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:49:59.858074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad508cd-273c-4ba3-9682-3c8ca30bf271 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:49:59.858363 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad508cd-273c-4ba3-9682-3c8ca30bf271 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:49:59.858660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad508cd-273c-4ba3-9682-3c8ca30bf271 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:49:59.868359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad508cd-273c-4ba3-9682-3c8ca30bf271 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:49:59.870001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad508cd-273c-4ba3-9682-3c8ca30bf271 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:49:59.878489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1236/2448] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:49:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:50:07.632432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3ab23-8ce6-4e53-9233-4d24ee5f6f9c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:07.632991 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3ab23-8ce6-4e53-9233-4d24ee5f6f9c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:07.633192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3ab23-8ce6-4e53-9233-4d24ee5f6f9c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:07.634497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1213/2449] 10.222.0.47 () {54 vars in 921 bytes} [Mon Jul 18 10:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:50:07.643663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:07.644154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:07.644275 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:07.663786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] MFA Rules not processed for user `f7b860ca751f4eb7bcc72617e05abd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:07.665217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:07.729569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1237/2450] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Jul 18 10:50:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:07.743087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3ab23-8ce6-4e53-9233-4d24ee5f6f9c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:07.743087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aac3ab23-8ce6-4e53-9233-4d24ee5f6f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:07.759982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3ab23-8ce6-4e53-9233-4d24ee5f6f9c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:07.765166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:07.765802 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:07.766088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:07.766369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:07.775808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:07.786785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:07.787856 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1214/2451] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:08.604406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:08.669056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7e672-2962-4c31-a4d8-50fc3e9dd947 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:08.673956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:08.674839 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:08.675124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:08.675467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] PATH_INFO: `/v3/users/80ff0f55699841c38e075e7709c30553` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:08.676992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ff0f55699841c38e075e7709c30553)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:08.679017 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:08.780062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:08.855270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ff0f55699841c38e075e7709c30553'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:08.862355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ff0f55699841c38e075e7709c30553'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:08.951981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ff0f55699841c38e075e7709c30553'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:08.952641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ff0f55699841c38e075e7709c30553'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:08.953801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ff0f55699841c38e075e7709c30553'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:08.973715 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1238/2452] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:08 2022] DELETE /identity/v3/users/80ff0f55699841c38e075e7709c30553 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:09.244618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:09.258761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562815d1-6456-4a35-a82e-2c364d164201 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:09.324260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:09.325147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:09.325434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:09.325713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] PATH_INFO: `/v3/projects/c6040be3467648939b7ff7138c05482c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:09.326879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6040be3467648939b7ff7138c05482c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:09.329068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:09.445922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6040be3467648939b7ff7138c05482c', 'request_id': 'req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.574218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6040be3467648939b7ff7138c05482c', 'request_id': 'req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.580808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6040be3467648939b7ff7138c05482c', 'request_id': 'req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.597179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] The token cache is being invalidate because project c6040be3467648939b7ff7138c05482c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:09.597580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.598974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.600721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1215/2453] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:09 2022] DELETE /identity/v3/projects/c6040be3467648939b7ff7138c05482c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:09.604900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:09.606339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:09.620197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a55e-66fa-4f54-9f33-4068b379e60d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:09.689601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:09.690483 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:09.690774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:09.691058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] PATH_INFO: `/v3/users/7215a95a5f0840d7921e9f601f0309d9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:09.692529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7215a95a5f0840d7921e9f601f0309d9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:09.694595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:09.912664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7215a95a5f0840d7921e9f601f0309d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.919527 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7215a95a5f0840d7921e9f601f0309d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.997082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7215a95a5f0840d7921e9f601f0309d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.998158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7215a95a5f0840d7921e9f601f0309d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:09.998798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b94c043-37e2-42f5-a1d2-51d4ca62467a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7215a95a5f0840d7921e9f601f0309d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:10.021518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1239/2454] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:09 2022] DELETE /identity/v3/users/7215a95a5f0840d7921e9f601f0309d9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:10.245382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:10.246958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:10.262431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea49296-3bcc-4526-bf68-ba1cbf5fd551 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:10.283644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:10.322233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:10.323097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:10.323386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:10.323666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] PATH_INFO: `/v3/projects/e85c29fed4c24951b047954a292a418a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:10.324901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85c29fed4c24951b047954a292a418a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:10.329143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:10.448298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85c29fed4c24951b047954a292a418a', 'request_id': 'req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:10.603227 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85c29fed4c24951b047954a292a418a', 'request_id': 'req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:10.609293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85c29fed4c24951b047954a292a418a', 'request_id': 'req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:10.620925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] The token cache is being invalidate because project e85c29fed4c24951b047954a292a418a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:10.621220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:10.621861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3257dc4e-72e5-4934-8845-4ce1c5eaeb7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:10.623410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1216/2455] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:10 2022] DELETE /identity/v3/projects/e85c29fed4c24951b047954a292a418a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:10.629931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:10.630221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:10.630507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:10.649935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:10.651163 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:10.669960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:10.716296 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1240/2456] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:10.725980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:10.726277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:10.726563 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:10.745747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:10.747044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:10.755266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1217/2457] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:10.759515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:10.769517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5df11-da42-41ba-8de9-1cb5c88b981e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:10.774685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:10.775348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:10.775639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:10.775946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:10.776700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:10.777024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:10.778638 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:10.790842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1241/2458] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:10.795121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:10.805305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d868c-336a-499b-a614-1f2efde6bbf6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:10.810061 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:10.810711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:10.810995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:10.811274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:10.811964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:10.813889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:10.917498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1218/2459] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:10 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:10.918633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:10.929520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ace2e-0048-4095-a8f3-a6471d685373 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:10.986760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:10.987468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:10.987753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:10.988108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:10.989354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:10.991366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:11.080981 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1242/2460] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:10 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:11.083323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:11.093730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc87b6-16fb-4fd8-99ed-393530db2d5e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:11.099183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:11.099883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:11.100167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:11.100448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:11.101220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:11.102845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:11.117839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1219/2461] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:11.120952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:11.132253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab135c3-8f2d-493f-bdc6-fc55dee9e528 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:11.136973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a045aa-b638-45ba-aecb-a1039c27a384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:11.137662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a045aa-b638-45ba-aecb-a1039c27a384 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:11.137935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a045aa-b638-45ba-aecb-a1039c27a384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:11.138219 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a045aa-b638-45ba-aecb-a1039c27a384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:11.138952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a045aa-b638-45ba-aecb-a1039c27a384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:11.140595 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a045aa-b638-45ba-aecb-a1039c27a384 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:11.151837 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1243/2462] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:11.156210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:11.166988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbbd90-2804-4725-a78e-097241cdcd40 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:11.171718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:11.172669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:11.173047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:11.173410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] PATH_INFO: `/v3/projects/1ac6415fe6874845bec8a4c2f76f0cfa/users/970fc4803dea41ed97f9a1f5fdc68488/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:11.189026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac6415fe6874845bec8a4c2f76f0cfa, user_id=970fc4803dea41ed97f9a1f5fdc68488, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:11.192418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:11.319022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1220/2463] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:11 2022] PUT /identity/v3/projects/1ac6415fe6874845bec8a4c2f76f0cfa/users/970fc4803dea41ed97f9a1f5fdc68488/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:11.329889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:11.330186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:11.330473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:11.359986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] MFA Rules not processed for user `970fc4803dea41ed97f9a1f5fdc68488`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:11.361565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:11.438109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1244/2464] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:11.450847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:11.452561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:11.513117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:11.517108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646d0c1-8b10-405e-a503-c162bef2bf1c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:11.521752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a7564-54ac-4956-9cf1-4c9000b8adb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:11.522518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a7564-54ac-4956-9cf1-4c9000b8adb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:11.522905 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a7564-54ac-4956-9cf1-4c9000b8adb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:11.523316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a7564-54ac-4956-9cf1-4c9000b8adb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:11.532297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a7564-54ac-4956-9cf1-4c9000b8adb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:11.533772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a7564-54ac-4956-9cf1-4c9000b8adb0 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:11.543319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1221/2465] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:11.570434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:11.594850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:11.639574 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d57e-7607-4bbb-abba-edc958f688a6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:11.645088 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:11.645918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:11.646203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:11.646480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] PATH_INFO: `/v3/users/970fc4803dea41ed97f9a1f5fdc68488` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:11.647832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=970fc4803dea41ed97f9a1f5fdc68488)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:11.650283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:11.911193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '970fc4803dea41ed97f9a1f5fdc68488'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:11.919161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '970fc4803dea41ed97f9a1f5fdc68488'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:12.046886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '970fc4803dea41ed97f9a1f5fdc68488'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:12.047513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '970fc4803dea41ed97f9a1f5fdc68488'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:12.048122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '970fc4803dea41ed97f9a1f5fdc68488'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:12.074672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1245/2466] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:11 2022] DELETE /identity/v3/users/970fc4803dea41ed97f9a1f5fdc68488 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:12.074875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:12.075094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:12.075380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:12.093236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:12.094469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:12.155194 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1222/2467] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:12.167092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:12.168627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:12.232899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9e9de-6c0b-41b8-968a-aa803921c637 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:12.236563 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afed926-872c-46d7-aa19-52c54c1dbb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:12.237251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afed926-872c-46d7-aa19-52c54c1dbb50 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:12.237555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afed926-872c-46d7-aa19-52c54c1dbb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:12.237855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afed926-872c-46d7-aa19-52c54c1dbb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:12.248373 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afed926-872c-46d7-aa19-52c54c1dbb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:12.250045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afed926-872c-46d7-aa19-52c54c1dbb50 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:12.262068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1246/2468] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:12.764669 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:12.767078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:12.782503 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5eb5-e356-460c-be3e-9fd89fcfbdbe None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:12.789295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:12.790621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:12.791072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:12.791487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] PATH_INFO: `/v3/projects/1ac6415fe6874845bec8a4c2f76f0cfa` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:12.793093 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac6415fe6874845bec8a4c2f76f0cfa)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:12.795186 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:12.934309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac6415fe6874845bec8a4c2f76f0cfa', 'request_id': 'req-313a75ed-c24c-4550-b027-beb3aa331d4f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:13.003036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac6415fe6874845bec8a4c2f76f0cfa', 'request_id': 'req-313a75ed-c24c-4550-b027-beb3aa331d4f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:13.011331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac6415fe6874845bec8a4c2f76f0cfa', 'request_id': 'req-313a75ed-c24c-4550-b027-beb3aa331d4f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:13.027162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] The token cache is being invalidate because project 1ac6415fe6874845bec8a4c2f76f0cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:13.027649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:13.028703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-313a75ed-c24c-4550-b027-beb3aa331d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:13.031277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1223/2469] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:12 2022] DELETE /identity/v3/projects/1ac6415fe6874845bec8a4c2f76f0cfa => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:13.036862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.037359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.037535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.056328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:13.057610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:13.101761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:13.125358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1247/2470] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:13.133526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b156fa-ed38-48ac-9504-af644535786b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.134029 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b156fa-ed38-48ac-9504-af644535786b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.134317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b156fa-ed38-48ac-9504-af644535786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.156478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-69b156fa-ed38-48ac-9504-af644535786b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:13.157858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-69b156fa-ed38-48ac-9504-af644535786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:13.168779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1224/2471] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:13.173233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.183655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcad19a-bcd3-4f65-a9ce-b3a235321a38 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.188355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.189039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.189325 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.189604 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.190353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:13.190679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.192936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.205400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1248/2472] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:13.210632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b156fa-ed38-48ac-9504-af644535786b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.220047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b156fa-ed38-48ac-9504-af644535786b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.224912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.229116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.229116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.229116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.229116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.229116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.310692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1225/2473] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:13 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:13.312614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.323429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce755c9a-5312-4a77-b921-94c99b682d9f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.382620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.383220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.383515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.383800 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.384495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.386500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.504352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1249/2474] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:13 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:13.508351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.518934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27189472-b6fc-45ba-99d8-51d7f710cad9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.525515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.526472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.526964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.527493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.528325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.530035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.545592 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1226/2475] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:13.546777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.557826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75773b4-1696-4e8b-903e-7e46e315d148 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.563104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fc9c5-00bc-4df6-81b5-997d301a0cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.563785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fc9c5-00bc-4df6-81b5-997d301a0cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.564132 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fc9c5-00bc-4df6-81b5-997d301a0cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.564418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fc9c5-00bc-4df6-81b5-997d301a0cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.565426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fc9c5-00bc-4df6-81b5-997d301a0cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.567113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fc9c5-00bc-4df6-81b5-997d301a0cbe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.577851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1250/2476] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:13.582688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.593049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fb20-9535-4f94-9092-0ffbb7035886 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.606141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.606141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.606141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.606141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] PATH_INFO: `/v3/projects/5d7ee0634e694265868916311d7fb4f1/users/cf3fce0b778f46df844a6be985c28b09/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.622825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7ee0634e694265868916311d7fb4f1, user_id=cf3fce0b778f46df844a6be985c28b09, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.623657 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.744195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1227/2477] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:13 2022] PUT /identity/v3/projects/5d7ee0634e694265868916311d7fb4f1/users/cf3fce0b778f46df844a6be985c28b09/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:13.749594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.749877 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.750162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.770038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:13.771258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:13.836130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1251/2478] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:13.845971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:13.848269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:13.861883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:13.908631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f7b7c-026a-489a-bd57-1cb70f1b065b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:13.915641 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce407c-5e39-4987-b747-7c3f6710fddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:13.916345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce407c-5e39-4987-b747-7c3f6710fddc service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:13.916628 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce407c-5e39-4987-b747-7c3f6710fddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:13.916937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce407c-5e39-4987-b747-7c3f6710fddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:13.927100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce407c-5e39-4987-b747-7c3f6710fddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:13.928516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce407c-5e39-4987-b747-7c3f6710fddc service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:13.938253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1228/2479] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:18.882861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:18.884308 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:18.899559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0487f1-80f7-4c63-a7ea-98f243c3bf22 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:18.904286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:18.905655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:18.906177 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:18.906468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:18.908582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:18.926066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:18.929006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:18.944290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1252/2480] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:20.105722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.107234 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.107646 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.152114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] MFA Rules not processed for user `cf3fce0b778f46df844a6be985c28b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:20.153387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:20.299444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1229/2481] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:20.304984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:20.314740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad040a-7294-47f9-9af9-433bd7b869a5 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:20.333023 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:20.333983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.334267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.334544 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.335245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:20.342697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:20.418937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1253/2482] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:20 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:20.425988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:20.449837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f5a0-1ab8-44aa-82e0-00a14fac4cec None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:20.515389 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:20.516356 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.516786 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.517101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.517879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:20.520402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:20.617195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1230/2483] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:20 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:20.621261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:20.632854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc373c-7efd-4dc0-987d-70841ccb1ba2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:20.638227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:20.639119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.639469 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.639816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.644616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:20.646530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:20.660789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1254/2484] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:20.664918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:20.677018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f2e51-0e85-4162-8bcf-0fc3eb5a29cb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:20.682373 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:20.683304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.683591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.683876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] PATH_INFO: `/v3/projects/d3126c02b94741949dc193f9d0508415/users/f175c299dfff4c8882c5c0c2657eb528/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.701151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3126c02b94741949dc193f9d0508415, user_id=f175c299dfff4c8882c5c0c2657eb528, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:20.704172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:20.775863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1231/2485] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:20 2022] PUT /identity/v3/projects/d3126c02b94741949dc193f9d0508415/users/f175c299dfff4c8882c5c0c2657eb528/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:20.780827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:20.787052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:20.792462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0199d-7535-4db8-acc2-3470f9ff433d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:20.863510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:20.864203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.864495 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.864815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.865617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:20.867360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:20.882675 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1255/2486] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:20.883045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:20.897798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc6b96-c666-45b5-82d8-0f1d0f6bdacf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:20.902620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:20.903549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:20.903835 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:20.904117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] PATH_INFO: `/v3/projects/d3126c02b94741949dc193f9d0508415/users/f175c299dfff4c8882c5c0c2657eb528/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:20.906160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3126c02b94741949dc193f9d0508415, user_id=f175c299dfff4c8882c5c0c2657eb528, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:20.911487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:20.915041 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:20.923188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1232/2487] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:20 2022] PUT /identity/v3/projects/d3126c02b94741949dc193f9d0508415/users/f175c299dfff4c8882c5c0c2657eb528/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:23.766819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:23.769776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:23.792075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02690-b50d-44aa-bea8-a7c573d05f0d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:23.923434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:23.924124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:23.924415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:23.925274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] PATH_INFO: `/v3/users/17e0c6def385441bbdde18c63b7d99e3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:23.927757 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e0c6def385441bbdde18c63b7d99e3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:23.932342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:24.238543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e0c6def385441bbdde18c63b7d99e3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:24.249676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e0c6def385441bbdde18c63b7d99e3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:24.402991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e0c6def385441bbdde18c63b7d99e3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:24.403799 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e0c6def385441bbdde18c63b7d99e3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:24.404594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d2703e4-db01-4a13-87dd-ebaf13cbde2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e0c6def385441bbdde18c63b7d99e3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:24.443115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1256/2488] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:23 2022] DELETE /identity/v3/users/17e0c6def385441bbdde18c63b7d99e3 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:24.808519 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:24.811385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:24.839188 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04cde8-937f-4d7d-9910-0ee0fd6efda9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:24.946944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:24.948915 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:24.949487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:24.949764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] PATH_INFO: `/v3/projects/9e4aa903be8d4afc91f368051333848b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:24.951360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4aa903be8d4afc91f368051333848b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:24.955724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:25.118217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4aa903be8d4afc91f368051333848b', 'request_id': 'req-c5119000-80a9-4f7c-9b5f-33871be11d7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:25.200934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4aa903be8d4afc91f368051333848b', 'request_id': 'req-c5119000-80a9-4f7c-9b5f-33871be11d7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:25.210327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4aa903be8d4afc91f368051333848b', 'request_id': 'req-c5119000-80a9-4f7c-9b5f-33871be11d7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:25.235677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] The token cache is being invalidate because project 9e4aa903be8d4afc91f368051333848b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:25.238925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:25.239102 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5119000-80a9-4f7c-9b5f-33871be11d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:25.247372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:25.247916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:25.253867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:25.255536 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1233/2489] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:24 2022] DELETE /identity/v3/projects/9e4aa903be8d4afc91f368051333848b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:25.292027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:25.293665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:25.312386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:25.381746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1257/2490] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:25.396912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:25.397200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:25.397484 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:25.424995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:25.426245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:25.435734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1234/2491] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:25.440135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:25.451418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46490-50e7-43de-abc3-d0afd5f5a825 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:25.457734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:25.458748 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:25.459192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:25.459628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:25.460576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:25.460928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:25.463043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:25.483217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:25.489319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1258/2492] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:50:25.494752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e074e5-6d84-409a-acff-aa9a1ac2b4f9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:25.499825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:25.502815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:25.502815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:25.502815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:25.502815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:25.504575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:25.634829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1235/2493] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:25 2022] POST /identity/v3/projects => generated 355 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:25.635172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:25.648948 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2d9ea-3033-43fc-8f5f-b0bf04ad9d46 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:25.769247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:25.772823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:25.772823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:25.772823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:25.772823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:25.776579 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:25.932791 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1259/2494] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:25 2022] POST /identity/v3/users => generated 278 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:25.934293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:25.954598 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb210c8-7194-4373-8c0d-d4d4cec2d155 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:25.963303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:25.964003 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:25.964333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:25.964642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:25.965416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:25.967035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:26.009362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1236/2495] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:26.012731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:26.024835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b039b-928d-4ed1-ad73-0fa897ede44d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:26.033137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462e164-3f95-4552-b59e-bd1bf7f631c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:26.035650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462e164-3f95-4552-b59e-bd1bf7f631c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:26.035650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462e164-3f95-4552-b59e-bd1bf7f631c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:26.035650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462e164-3f95-4552-b59e-bd1bf7f631c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:26.036389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462e164-3f95-4552-b59e-bd1bf7f631c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:26.038173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462e164-3f95-4552-b59e-bd1bf7f631c1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:26.056367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:26.057687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1260/2496] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:26.075063 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6090-680f-4a0e-a536-4c267fdff358 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:26.080948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:26.083292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:26.083600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:26.083887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] PATH_INFO: `/v3/projects/aed1daec0963481f9e2dd4e37ef11d74/users/b7252b9f030d4202be44c46bab04f78f/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:26.102700 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed1daec0963481f9e2dd4e37ef11d74, user_id=b7252b9f030d4202be44c46bab04f78f, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:26.105582 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:26.187942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:26.188252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:26.188549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:26.190217 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1237/2497] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:26 2022] PUT /identity/v3/projects/aed1daec0963481f9e2dd4e37ef11d74/users/b7252b9f030d4202be44c46bab04f78f/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:26.237832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] MFA Rules not processed for user `b7252b9f030d4202be44c46bab04f78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:26.239119 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:26.348490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1261/2498] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:26 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:26.350970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:26.411829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:26.489753 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10e334-6a0d-4259-8b60-df4df2976967 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:26.500603 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:26.502375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:26.502375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:26.502375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:26.504630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:26.509286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:26.752673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1238/2499] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:26 2022] POST /identity/v3/projects => generated 357 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:26.757005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:26.769504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0164721-f351-411c-a478-9a8b14f59623 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:26.782346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:26.868034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:26.869071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:26.869444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:26.869804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:26.870583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:26.872764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:26.990372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1262/2500] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:26 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:26.990528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.001253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6b2d-74ad-43be-9f10-1b83e2d505d3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.006743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.007440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.007719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.008199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.008967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:27.011001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:27.027355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1239/2501] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:27.031482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.043964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c078-bcee-4a1a-b6ab-eab191f3770b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.048500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.049637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.050039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.050601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] PATH_INFO: `/v3/projects/ed10915ee7934c76a27a6fc56ccf24b5/users/e715e748823e4b5082b2864249804f5c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.070189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed10915ee7934c76a27a6fc56ccf24b5, user_id=e715e748823e4b5082b2864249804f5c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:27.075164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:27.201646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1263/2502] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:27 2022] PUT /identity/v3/projects/ed10915ee7934c76a27a6fc56ccf24b5/users/e715e748823e4b5082b2864249804f5c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:27.205636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.220256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bffd85-c325-4f1e-b0c1-36fd0bad5425 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.330513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f36d3-852c-4409-8ec3-252a0fbf1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.331514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f36d3-852c-4409-8ec3-252a0fbf1e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.331798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f36d3-852c-4409-8ec3-252a0fbf1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.332146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f36d3-852c-4409-8ec3-252a0fbf1e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.332977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f36d3-852c-4409-8ec3-252a0fbf1e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:27.334781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f36d3-852c-4409-8ec3-252a0fbf1e60 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:27.346577 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1240/2503] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:27.351309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.366588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf50baa-feca-4074-84cf-2e45c476c5d4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.372784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.375238 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.375722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.376284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] PATH_INFO: `/v3/projects/ed10915ee7934c76a27a6fc56ccf24b5/users/e715e748823e4b5082b2864249804f5c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.379433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed10915ee7934c76a27a6fc56ccf24b5, user_id=e715e748823e4b5082b2864249804f5c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:27.382868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:27.397302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1264/2504] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:27 2022] PUT /identity/v3/projects/ed10915ee7934c76a27a6fc56ccf24b5/users/e715e748823e4b5082b2864249804f5c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:27.409936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43234-a298-463f-971b-044a77607a11 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.414501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43234-a298-463f-971b-044a77607a11 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.416144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43234-a298-463f-971b-044a77607a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.452868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b9f43234-a298-463f-971b-044a77607a11 None None] MFA Rules not processed for user `e715e748823e4b5082b2864249804f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:27.455298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b9f43234-a298-463f-971b-044a77607a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:27.552483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1241/2505] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:27 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:27.560089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.562230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:27.650971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19208bb7-2dc0-4aea-9c53-fce4003683fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.651369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.651664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.651948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.655069 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.655901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.656290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.656713 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.669613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:27.671255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:27.683874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] MFA Rules not processed for user `f175c299dfff4c8882c5c0c2657eb528`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:27.685726 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:27.693333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1265/2506] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:27.767729 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1242/2507] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:27.783756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.785851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:27.849824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:27.870807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf54008-fd5e-4460-b9c5-7ba251d32623 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.876422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.877115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.877403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.877684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.879639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:27.885124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:27.936315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:27.946602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81c83d-09e9-4c3e-9aac-d3ddcdeb035f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:27.950475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:27.951124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:27.951249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:27.951409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:27.951687 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:27.953842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:27.968444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1266/2508] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:28.026758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:28.029074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:28.042087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1243/2509] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:37.071278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:37.073109 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:37.138210 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7e15-d67a-4ca1-b17d-a15dc1973427 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:37.141945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:37.142600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:37.142919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:37.143188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:37.145387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:37.216518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:37.218227 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:37.229194 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1267/2510] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:38.631852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=54 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:42.043107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:42.044823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:42.060115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9751-3f78-4a40-bf52-04688a05b4e9 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:42.065714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:42.066944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:42.067238 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:42.067527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] PATH_INFO: `/v3/users/46d0d621460b4e5b877ef421d3503340` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:42.068845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d0d621460b4e5b877ef421d3503340)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:42.070877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:42.463772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d0d621460b4e5b877ef421d3503340'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:42.483182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d0d621460b4e5b877ef421d3503340'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:42.491511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d0d621460b4e5b877ef421d3503340'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:42.612153 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d0d621460b4e5b877ef421d3503340'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:42.612814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-17e898f9-9252-47b1-b056-3d1e68b09101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d0d621460b4e5b877ef421d3503340'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:42.620264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1244/2511] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:42 2022] DELETE /identity/v3/users/46d0d621460b4e5b877ef421d3503340 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:42.902383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:42.914916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b20a4-2669-4f27-b169-36e499ddcfc4 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:42.967554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:42.975420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:42.976255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:42.976549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:42.976905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] PATH_INFO: `/v3/projects/f76bdb50a5c540e9aeb2099b993d334b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:42.978102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76bdb50a5c540e9aeb2099b993d334b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:42.980560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.124829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76bdb50a5c540e9aeb2099b993d334b', 'request_id': 'req-30bc967c-dd00-4bd9-9006-7e344de4767c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:43.131249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76bdb50a5c540e9aeb2099b993d334b', 'request_id': 'req-30bc967c-dd00-4bd9-9006-7e344de4767c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:43.257437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76bdb50a5c540e9aeb2099b993d334b', 'request_id': 'req-30bc967c-dd00-4bd9-9006-7e344de4767c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:43.269740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] The token cache is being invalidate because project f76bdb50a5c540e9aeb2099b993d334b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:43.270251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:43.271622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30bc967c-dd00-4bd9-9006-7e344de4767c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:43.273553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1268/2512] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:42 2022] DELETE /identity/v3/projects/f76bdb50a5c540e9aeb2099b993d334b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:43.279730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.280231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.280426 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.299850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:43.304388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:43.383259 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1245/2513] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:43.384851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.385147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.385434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.404909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:43.406256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:43.415404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1269/2514] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:43.419763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:43.430969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf632-e24b-4522-beca-f877bf76b051 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:43.435511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:43.436498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.436811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.437092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.437822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:43.438131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:43.440493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.452338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1246/2515] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:43.457060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:43.466406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e3814-e1cb-45dc-8fdd-a4d0b540df17 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:43.471136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:43.471845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.472136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.472411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.473229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:43.475112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.573926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1270/2516] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:43 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:43.575191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:43.586543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0674-3e12-471c-9e6a-63b3ca17f553 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:43.646690 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:43.647355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.647636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.647910 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.648589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:43.650611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.657246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:43.750868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1247/2517] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:43 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:43.753855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:43.764233 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159fe43-99cc-4482-8a6f-c2f5dca962e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:43.769788 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:43.770452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.770746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.771039 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.771778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:43.773519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.784251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1271/2518] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:43.788622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:43.798811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644244fb-12c4-41bb-bd82-a7d8d28972d6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:43.805254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa683-d0e7-4f88-addd-acd6ebafd031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:43.805922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa683-d0e7-4f88-addd-acd6ebafd031 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.806212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa683-d0e7-4f88-addd-acd6ebafd031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.806633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa683-d0e7-4f88-addd-acd6ebafd031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.807457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa683-d0e7-4f88-addd-acd6ebafd031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:43.809057 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa683-d0e7-4f88-addd-acd6ebafd031 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.819881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1248/2519] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:43.823834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:43.834512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcda67-7aec-4a9f-91c6-7cfa2b3e3901 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:43.839615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:43.840504 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.840821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.841097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] PATH_INFO: `/v3/projects/06d05d8aba8b42309e0bb1a66c8ecb76/users/6389b95e5fbc4bbaaffed714618ffb72/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.856876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d05d8aba8b42309e0bb1a66c8ecb76, user_id=6389b95e5fbc4bbaaffed714618ffb72, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:43.859575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:43.950405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1272/2520] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:43 2022] PUT /identity/v3/projects/06d05d8aba8b42309e0bb1a66c8ecb76/users/6389b95e5fbc4bbaaffed714618ffb72/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:43.960670 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:43.961005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:43.961326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:43.990509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] MFA Rules not processed for user `6389b95e5fbc4bbaaffed714618ffb72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:43.991816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:44.059667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1249/2521] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:44.064320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.133697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692e4f-ddea-41e9-98fc-3d2852e9dd8d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.138912 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.139697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.139984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.140274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.141049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.144382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.277114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1273/2522] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:44 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:44.281513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.293132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f584246-5881-4251-9aab-997838e2ba0d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.351182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.351772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.352058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.352336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.353091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.355005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.425431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1250/2523] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:44 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:44.429303 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.439737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876373d-f0dc-458a-beab-5bdf006d4c04 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.444594 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.445270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.445561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.445845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.446578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.448192 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.449507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:44.461207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1274/2524] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:44.465294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.476200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8238dd-a60a-4cd7-b767-bf17e6e0cc1a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.480748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.481620 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.481904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.482679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] PATH_INFO: `/v3/projects/8386e7ea4310462e8dcd1b50190ac248/users/3ee073f953aa409099b993e10167b314/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.497427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8386e7ea4310462e8dcd1b50190ac248, user_id=3ee073f953aa409099b993e10167b314, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.500103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.603424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1251/2525] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:44 2022] PUT /identity/v3/projects/8386e7ea4310462e8dcd1b50190ac248/users/3ee073f953aa409099b993e10167b314/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:44.607491 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.617577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7673a66-9d1f-4cdf-a448-30d69524227a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.723344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.725101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:44.764821 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:44.787663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02560d-61c9-45fb-8501-63f15349b873 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.791355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.792005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.792293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.792568 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.794569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:44.798225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.798899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.799727 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.800011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.800774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.802362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.808030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.809499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ca9d8-4056-41ac-a26e-df58b0b15c9c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.812517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1275/2526] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:44.816441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.819887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1252/2527] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:44.826208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2585b5c-13c1-42ae-b424-33700a5ce3fe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:44.831886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:44.832785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.833079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.833362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] PATH_INFO: `/v3/projects/8386e7ea4310462e8dcd1b50190ac248/users/3ee073f953aa409099b993e10167b314/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.835380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8386e7ea4310462e8dcd1b50190ac248, user_id=3ee073f953aa409099b993e10167b314, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:44.838061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:44.849315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1276/2528] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:44 2022] PUT /identity/v3/projects/8386e7ea4310462e8dcd1b50190ac248/users/3ee073f953aa409099b993e10167b314/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:44.859483 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:44.859706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:44.859996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:44.886546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] MFA Rules not processed for user `3ee073f953aa409099b993e10167b314`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:44.887765 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:44.957529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1253/2529] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:44.969097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:44.970943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:45.030870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074da845-715c-471c-a0a2-240d6945aa73 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:45.035130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6987936-c77e-49cb-9aca-df719636d60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:45.035774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987936-c77e-49cb-9aca-df719636d60e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:45.036062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987936-c77e-49cb-9aca-df719636d60e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:45.036342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987936-c77e-49cb-9aca-df719636d60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:45.045694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6987936-c77e-49cb-9aca-df719636d60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:45.047137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6987936-c77e-49cb-9aca-df719636d60e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:45.056669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1277/2530] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:46.287879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:46.354606 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66e869-38d3-40bf-bad0-501b6d085ccb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:46.359454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:46.360374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:46.360656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:46.360965 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] PATH_INFO: `/v3/users/6389b95e5fbc4bbaaffed714618ffb72` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:46.362485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6389b95e5fbc4bbaaffed714618ffb72)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:46.364428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:46.570116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6389b95e5fbc4bbaaffed714618ffb72'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:46.582698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:46.584272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6389b95e5fbc4bbaaffed714618ffb72'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:46.591193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6389b95e5fbc4bbaaffed714618ffb72'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:46.722955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6389b95e5fbc4bbaaffed714618ffb72'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:46.724013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6389b95e5fbc4bbaaffed714618ffb72'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:46.731281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1254/2531] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:46 2022] DELETE /identity/v3/users/6389b95e5fbc4bbaaffed714618ffb72 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:46.735763 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:46.736258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:46.736391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:46.755127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:46.756352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:46.825436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1278/2532] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:46.836173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:46.837816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:46.903017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb62d22-d245-4e9e-bbd8-2158ca3077e6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:46.907416 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:46.908161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:46.909076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:46.909485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:46.920461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:46.923069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:46.934086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1255/2533] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:47.462486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:47.463930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:47.483493 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103120e-9edc-4adf-9dd0-16562fe4cf26 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:47.491571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:47.492796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:47.493000 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:47.493564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] PATH_INFO: `/v3/projects/06d05d8aba8b42309e0bb1a66c8ecb76` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:47.494820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d05d8aba8b42309e0bb1a66c8ecb76)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:47.496866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:47.515540 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:47.629357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d05d8aba8b42309e0bb1a66c8ecb76', 'request_id': 'req-8834bdec-da48-4c8f-b180-aec4f827fcc4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:47.635294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d05d8aba8b42309e0bb1a66c8ecb76', 'request_id': 'req-8834bdec-da48-4c8f-b180-aec4f827fcc4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:47.706364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d05d8aba8b42309e0bb1a66c8ecb76', 'request_id': 'req-8834bdec-da48-4c8f-b180-aec4f827fcc4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:47.724969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] The token cache is being invalidate because project 06d05d8aba8b42309e0bb1a66c8ecb76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:47.725286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:47.725930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8834bdec-da48-4c8f-b180-aec4f827fcc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:47.728171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1279/2534] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:47 2022] DELETE /identity/v3/projects/06d05d8aba8b42309e0bb1a66c8ecb76 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:47.732342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:47.742792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a0347-e426-4c6a-aa59-5ff79f46ff3f service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:47.812666 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:47.813599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:47.813888 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:47.814177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] PATH_INFO: `/v3/users/3ee073f953aa409099b993e10167b314` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:47.815616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee073f953aa409099b993e10167b314)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:47.817640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:48.073576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee073f953aa409099b993e10167b314'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:48.092085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee073f953aa409099b993e10167b314'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:48.108630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee073f953aa409099b993e10167b314'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:48.206486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee073f953aa409099b993e10167b314'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:48.207135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee073f953aa409099b993e10167b314'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:48.217523 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1256/2535] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:47 2022] DELETE /identity/v3/users/3ee073f953aa409099b993e10167b314 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:48.428378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:48.428970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:48.429247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:48.448573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] MFA Rules not processed for user `cf3fce0b778f46df844a6be985c28b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:48.450012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:48.523700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1280/2536] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:48.849179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:48.852510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:48.853891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:48.855388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:48.856570 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:48.944023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c481e-ebbe-4bb2-a126-0ff722f58a36 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:48.948838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:48.949893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:48.950182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:48.950293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b2fe8-cfeb-4628-a947-83395931cf9a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:48.950461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] PATH_INFO: `/v3/projects/8386e7ea4310462e8dcd1b50190ac248` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:48.951840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8386e7ea4310462e8dcd1b50190ac248)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:48.954255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:48.956459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:48.957388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:48.957676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:48.957955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] PATH_INFO: `/v3/users/b7252b9f030d4202be44c46bab04f78f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:48.959258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7252b9f030d4202be44c46bab04f78f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:48.961204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:49.209652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7252b9f030d4202be44c46bab04f78f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.210166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8386e7ea4310462e8dcd1b50190ac248', 'request_id': 'req-3abefdc7-9654-4dde-b3b3-279c6cbfc288'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.215756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:49.218491 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7252b9f030d4202be44c46bab04f78f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.259417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8386e7ea4310462e8dcd1b50190ac248', 'request_id': 'req-3abefdc7-9654-4dde-b3b3-279c6cbfc288'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.265723 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8386e7ea4310462e8dcd1b50190ac248', 'request_id': 'req-3abefdc7-9654-4dde-b3b3-279c6cbfc288'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.278340 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] The token cache is being invalidate because project 8386e7ea4310462e8dcd1b50190ac248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:49.278795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.279454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.281076 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1257/2537] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:48 2022] DELETE /identity/v3/projects/8386e7ea4310462e8dcd1b50190ac248 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:49.285109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:49.285109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:49.334154 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7252b9f030d4202be44c46bab04f78f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.334988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7252b9f030d4202be44c46bab04f78f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.335751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58ec0c19-100e-4a7e-bbbe-5fa67624d64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7252b9f030d4202be44c46bab04f78f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.348375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abefdc7-9654-4dde-b3b3-279c6cbfc288 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:49.353035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:49.353923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.354208 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.354484 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] PATH_INFO: `/v3/users/df7a3101955b481f9a582168731ae76c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.355706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7a3101955b481f9a582168731ae76c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:49.358765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1281/2538] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:48 2022] DELETE /identity/v3/users/b7252b9f030d4202be44c46bab04f78f => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:49.358765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684f04e-f256-498b-946d-66c5b040c0ff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.358765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684f04e-f256-498b-946d-66c5b040c0ff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.358765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684f04e-f256-498b-946d-66c5b040c0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.359122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:49.376038 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c684f04e-f256-498b-946d-66c5b040c0ff None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:49.377302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c684f04e-f256-498b-946d-66c5b040c0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:49.444362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1282/2539] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:49.444362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39181c3-25d3-4b90-8646-0e633f4b4a4f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.444362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39181c3-25d3-4b90-8646-0e633f4b4a4f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.444362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39181c3-25d3-4b90-8646-0e633f4b4a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.459448 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c39181c3-25d3-4b90-8646-0e633f4b4a4f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:49.460692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c39181c3-25d3-4b90-8646-0e633f4b4a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:49.468597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1283/2540] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:49.474809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09071-7059-4760-8377-ca33252426c2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.474809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09071-7059-4760-8377-ca33252426c2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.474809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09071-7059-4760-8377-ca33252426c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.489743 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-07c09071-7059-4760-8377-ca33252426c2 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:49.490972 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07c09071-7059-4760-8377-ca33252426c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:49.499537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1284/2541] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:49.502734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09071-7059-4760-8377-ca33252426c2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:49.502734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07c09071-7059-4760-8377-ca33252426c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:49.558545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09071-7059-4760-8377-ca33252426c2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:49.562124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:49.562513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7a3101955b481f9a582168731ae76c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.562772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.563067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.563347 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.572810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:49.574246 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:49.574619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7a3101955b481f9a582168731ae76c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.582706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7a3101955b481f9a582168731ae76c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.585534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1285/2542] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:49.590954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:49.595332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81045e9-171e-4c8e-9f86-6993326cb93e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:49.601269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:49.601852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.602138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.602417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.603143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:49.603454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:49.605137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:49.617032 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1286/2543] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:49.621384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:49.631403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cedf7d-eb4e-47e6-8e92-ba43f0f923b2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:49.636059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:49.636729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.637020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.637297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.637979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:49.640199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:49.687913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7a3101955b481f9a582168731ae76c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.688533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7a3101955b481f9a582168731ae76c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:49.698943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1258/2544] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:49 2022] DELETE /identity/v3/users/df7a3101955b481f9a582168731ae76c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:49.768206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1287/2545] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:49 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:49.772334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:49.774009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:49.860800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f280e7-2e83-4aeb-af24-5ede1cb6b040 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:49.899452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:49.922040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:49.922730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:49.923022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:49.923307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:49.924074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:49.926218 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.002552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1259/2546] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:49 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:50.009010 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.021012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864903e-bb86-4e32-84b8-5f08b9502676 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.025685 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.026535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.026820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.027182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.027861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.029535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.041292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1288/2547] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:50.045282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.055609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a0458-6f24-467d-9f1c-e36ace2d064c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.065371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.065371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.065371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.065371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.066792 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.068575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.079860 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1260/2548] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:50.085435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.096751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.098106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743de19-de1d-4ac4-86c7-c7b2f9b7c97a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.098576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.104265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.105121 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.105387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.105663 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] PATH_INFO: `/v3/projects/196fbe2e602e427a8d36d5a3895ccb75/users/7066ca96d7864616b4a6032e16966a54/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.114476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057b57-b6fe-4aaa-a5d6-0525b0857e00 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.122660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.122660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.122660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.122921 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=196fbe2e602e427a8d36d5a3895ccb75, user_id=7066ca96d7864616b4a6032e16966a54, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.123023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] PATH_INFO: `/v3/projects/ad323d7f7c484b5bb161d92791c9c7d7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.124141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad323d7f7c484b5bb161d92791c9c7d7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.125328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.126066 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.243155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1289/2549] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:50 2022] PUT /identity/v3/projects/196fbe2e602e427a8d36d5a3895ccb75/users/7066ca96d7864616b4a6032e16966a54/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:50.244058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.255534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.282546 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad323d7f7c484b5bb161d92791c9c7d7', 'request_id': 'req-09ca6747-36fb-4619-8b93-e52734d81d3d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.321327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7888f-3050-4632-a84d-a224f0a64630 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.354898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad323d7f7c484b5bb161d92791c9c7d7', 'request_id': 'req-09ca6747-36fb-4619-8b93-e52734d81d3d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.361229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad323d7f7c484b5bb161d92791c9c7d7', 'request_id': 'req-09ca6747-36fb-4619-8b93-e52734d81d3d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.373284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] The token cache is being invalidate because project ad323d7f7c484b5bb161d92791c9c7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:50.373579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.374246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-09ca6747-36fb-4619-8b93-e52734d81d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.375893 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1261/2550] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:50 2022] DELETE /identity/v3/projects/ad323d7f7c484b5bb161d92791c9c7d7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:50.378012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d9fa-b566-424e-8cde-da66317bece3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.378303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d9fa-b566-424e-8cde-da66317bece3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.378586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d9fa-b566-424e-8cde-da66317bece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.380628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.381532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.381816 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.382188 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] PATH_INFO: `/v3/projects/aed1daec0963481f9e2dd4e37ef11d74` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.383473 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed1daec0963481f9e2dd4e37ef11d74)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.385426 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.404713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-02e7d9fa-b566-424e-8cde-da66317bece3 None None] MFA Rules not processed for user `7066ca96d7864616b4a6032e16966a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:50.405938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-02e7d9fa-b566-424e-8cde-da66317bece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.441967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:50.470206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1262/2551] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:50.472342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.472640 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.472952 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.492307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:50.493572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.508310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed1daec0963481f9e2dd4e37ef11d74', 'request_id': 'req-fb1f047a-5d44-46c2-b230-25d251d9ec45'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.515077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed1daec0963481f9e2dd4e37ef11d74', 'request_id': 'req-fb1f047a-5d44-46c2-b230-25d251d9ec45'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.555474 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1263/2552] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:50.556518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.557808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.569432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12832478-2330-40d1-a80a-dc1d503f3c57 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.583039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.583039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.583039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.583039 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.583615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.583615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.602459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed1daec0963481f9e2dd4e37ef11d74', 'request_id': 'req-fb1f047a-5d44-46c2-b230-25d251d9ec45'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.614160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] The token cache is being invalidate because project aed1daec0963481f9e2dd4e37ef11d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:50.614464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.615100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb1f047a-5d44-46c2-b230-25d251d9ec45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.616661 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1290/2553] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:50 2022] DELETE /identity/v3/projects/aed1daec0963481f9e2dd4e37ef11d74 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:50.617898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab537-f96c-4501-9cff-641574812081 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.618189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab537-f96c-4501-9cff-641574812081 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.618471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab537-f96c-4501-9cff-641574812081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.635844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e2fab537-f96c-4501-9cff-641574812081 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:50.637079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e2fab537-f96c-4501-9cff-641574812081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.644884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1291/2554] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:50.645648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fab537-f96c-4501-9cff-641574812081 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.647067 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e2fab537-f96c-4501-9cff-641574812081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.660035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fab537-f96c-4501-9cff-641574812081 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.664548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.665419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.665706 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.665980 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] PATH_INFO: `/v3/users/e715e748823e4b5082b2864249804f5c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.667158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e715e748823e4b5082b2864249804f5c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.669134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.733144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1264/2555] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:50 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:50.734070 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.795123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc838b2-98e5-4289-a6c2-3334e5b846f8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.800899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.801567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.801855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.802141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.802875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:50.803189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.804923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.816554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1265/2556] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:50.817386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.819054 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:50.841087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4d709-8154-45bf-a707-d3049c5eac61 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.841087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.841087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.841087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.841585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.841922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.843855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:50.866968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:50.946213 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1266/2557] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:50 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:50.947866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:50.954895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e715e748823e4b5082b2864249804f5c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.959415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe125c-bdb3-4b1f-a33f-0d1699252886 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:50.963415 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e715e748823e4b5082b2864249804f5c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:50.964526 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:50.965277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:50.965638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:50.966024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:50.966771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:50.968825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.030937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e715e748823e4b5082b2864249804f5c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.031699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e715e748823e4b5082b2864249804f5c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.032320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e715e748823e4b5082b2864249804f5c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.040001 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1267/2558] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:50 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:51.041144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.044115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:51.053113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1292/2559] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:50 2022] DELETE /identity/v3/users/e715e748823e4b5082b2864249804f5c => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:51.054028 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.055577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:51.077552 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:51.121297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0e23-7b4a-4880-a2ab-7960d34185ec admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.127237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.128533 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.129514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.129514 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.129867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67c3c-bb66-4f94-ac4d-e97cd2e560c3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.130143 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.132266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.135455 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.136155 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.136445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.136747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.137489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.139549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.149819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1268/2560] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:51.151144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.162142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59603-f08b-45f0-a54d-31fd39658dc3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.167899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.168839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.169122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.169397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] PATH_INFO: `/v3/projects/a4e0106b85cc4d06aa9967bcae73e9f7/users/8ad5d804816942f9b33d2eac8085f255/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.184387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e0106b85cc4d06aa9967bcae73e9f7, user_id=8ad5d804816942f9b33d2eac8085f255, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.187085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.258465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1293/2561] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:51 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:51.262904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.277936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04add9ba-daa8-45fc-b232-98d43616d179 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.280839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:51.286229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.287065 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.287379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.287676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.296629 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1269/2562] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:51 2022] PUT /identity/v3/projects/a4e0106b85cc4d06aa9967bcae73e9f7/users/8ad5d804816942f9b33d2eac8085f255/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:51.296961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.296961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.300207 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.304768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1294/2563] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:51.309927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.321115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05df67-e12a-4db7-a6e9-b282ac5c4391 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.390101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.391546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.392182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.393207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.394442 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.396555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.406548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6436d-e398-45aa-8674-1d8622e80936 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.412792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1295/2564] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:51.414360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.415099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.415362 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.415469 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.415780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.416497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.418121 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.430267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1270/2565] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:51.431327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f0d3-e8dc-4c51-8822-40a61c795c8e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.436952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.437394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.437688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.437971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] PATH_INFO: `/v3/projects/f6898f6872fd4bd381b8f76848ad29d8/users/9f065e19b02a402fa63b9fa7cbea4f74/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.440315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.450189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24120d-99b6-484c-9952-abadc06c521f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.453969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6898f6872fd4bd381b8f76848ad29d8, user_id=9f065e19b02a402fa63b9fa7cbea4f74, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.455988 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e039971-9dbe-426c-bbcd-10776956ea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.456892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e039971-9dbe-426c-bbcd-10776956ea9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.457187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e039971-9dbe-426c-bbcd-10776956ea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.457506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e039971-9dbe-426c-bbcd-10776956ea9c admin admin] PATH_INFO: `/v3/projects/a4e0106b85cc4d06aa9967bcae73e9f7/users/8ad5d804816942f9b33d2eac8085f255/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.457613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.459487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e039971-9dbe-426c-bbcd-10776956ea9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e0106b85cc4d06aa9967bcae73e9f7, user_id=8ad5d804816942f9b33d2eac8085f255, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.462236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e039971-9dbe-426c-bbcd-10776956ea9c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.473292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1271/2566] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:51 2022] PUT /identity/v3/projects/a4e0106b85cc4d06aa9967bcae73e9f7/users/8ad5d804816942f9b33d2eac8085f255/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:51.483518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b71db-87b7-4060-b11e-751387d603d9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.483818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b71db-87b7-4060-b11e-751387d603d9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.484106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b71db-87b7-4060-b11e-751387d603d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.488098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:51.514953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-298b71db-87b7-4060-b11e-751387d603d9 None None] MFA Rules not processed for user `8ad5d804816942f9b33d2eac8085f255`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:51.516173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-298b71db-87b7-4060-b11e-751387d603d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:51.572714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1296/2567] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:51 2022] PUT /identity/v3/projects/f6898f6872fd4bd381b8f76848ad29d8/users/9f065e19b02a402fa63b9fa7cbea4f74/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:51.572714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.572714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:51.591418 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1272/2568] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:51.601185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.601185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.601185 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.620444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] MFA Rules not processed for user `9f065e19b02a402fa63b9fa7cbea4f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:51.621705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:51.630573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04f8de-e25d-4db2-8bbf-1f0458dd2c9c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.644309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.644309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.644309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.644309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] PATH_INFO: `/v3/projects/ed10915ee7934c76a27a6fc56ccf24b5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.644781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed10915ee7934c76a27a6fc56ccf24b5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.644781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.689621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1273/2569] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:51.690667 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.691944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:51.750257 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971df498-1cdc-466c-9e51-b13b28796e09 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.754761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.755408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.755693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.755973 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.803782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:51.820263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.820436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed10915ee7934c76a27a6fc56ccf24b5', 'request_id': 'req-6897cfaa-f37a-49ce-8cef-0dd86468e20d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.821756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.827059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed10915ee7934c76a27a6fc56ccf24b5', 'request_id': 'req-6897cfaa-f37a-49ce-8cef-0dd86468e20d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.834075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1274/2570] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:51.835069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.845648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030089-9dab-46f9-b7b6-737e51148b30 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:51.910866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de53ba1-6ea0-4614-9ad3-278357775f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:51.911663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de53ba1-6ea0-4614-9ad3-278357775f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:51.912161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de53ba1-6ea0-4614-9ad3-278357775f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:51.913303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de53ba1-6ea0-4614-9ad3-278357775f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:51.913303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de53ba1-6ea0-4614-9ad3-278357775f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:51.915807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de53ba1-6ea0-4614-9ad3-278357775f16 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:51.972152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed10915ee7934c76a27a6fc56ccf24b5', 'request_id': 'req-6897cfaa-f37a-49ce-8cef-0dd86468e20d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.983666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] The token cache is being invalidate because project ed10915ee7934c76a27a6fc56ccf24b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:51.983962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.985959 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:51.990239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1297/2571] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:51 2022] DELETE /identity/v3/projects/ed10915ee7934c76a27a6fc56ccf24b5 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:51.991072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:51.992785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.006389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897cfaa-f37a-49ce-8cef-0dd86468e20d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.011257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.012139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.012425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.012738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] PATH_INFO: `/v3/users/7066ca96d7864616b4a6032e16966a54` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.013962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7066ca96d7864616b4a6032e16966a54)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.016001 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.042403 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1275/2572] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:51 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:52.043881 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.044311 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.044697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.062979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:52.064200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.124145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1276/2573] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:52.125588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.127178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.139535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73636328-1cc2-41dd-8458-1066d89cbca4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.146104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aec6d-6800-4026-a55f-dc2da9d2fde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.146992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aec6d-6800-4026-a55f-dc2da9d2fde5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.147351 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aec6d-6800-4026-a55f-dc2da9d2fde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.147780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aec6d-6800-4026-a55f-dc2da9d2fde5 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.148501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647aec6d-6800-4026-a55f-dc2da9d2fde5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.150516 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647aec6d-6800-4026-a55f-dc2da9d2fde5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.164589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:52.231810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7066ca96d7864616b4a6032e16966a54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:52.240725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7066ca96d7864616b4a6032e16966a54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:52.253456 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1277/2574] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:52.257538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.257834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.258116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.277636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:52.278889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.287685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1278/2575] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:52.288709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.298235 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528c1a1-5042-459f-a4b3-4a3875a361c6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.303768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.304432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.304754 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.305040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.305774 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.307365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.319036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1279/2576] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:52.319036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.331983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcaf12-00b2-4ade-a64f-5018b3bbe220 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.337094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.337763 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.338060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.338347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.338718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7066ca96d7864616b4a6032e16966a54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:52.339095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:52.339583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.341013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7066ca96d7864616b4a6032e16966a54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:52.342135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7066ca96d7864616b4a6032e16966a54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:52.342337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:52.346733 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.350769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:52.360508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1280/2577] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:52.360508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.363827 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.364536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1298/2578] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:51 2022] DELETE /identity/v3/users/7066ca96d7864616b4a6032e16966a54 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:52.365383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.369006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.384727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e17b3-b516-4112-979d-6834cb510a68 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.441997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8ab6e-1f74-458c-9550-0068758319a3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.447375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c196e-e802-43f1-9037-ea46397d5c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.447882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.448591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c196e-e802-43f1-9037-ea46397d5c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.448732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.448908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c196e-e802-43f1-9037-ea46397d5c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.449083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.449182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c196e-e802-43f1-9037-ea46397d5c30 admin admin] PATH_INFO: `/v3/projects/1eb1c0fb84394a80936f23e6ee57b842/users/2bc7c620527d4d5d870fd28f6c0d5a7c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.449486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.450376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.452716 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.468301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c196e-e802-43f1-9037-ea46397d5c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb1c0fb84394a80936f23e6ee57b842, user_id=2bc7c620527d4d5d870fd28f6c0d5a7c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.471578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c196e-e802-43f1-9037-ea46397d5c30 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.564034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1281/2579] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:52 2022] PUT /identity/v3/projects/1eb1c0fb84394a80936f23e6ee57b842/users/2bc7c620527d4d5d870fd28f6c0d5a7c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:52.565487 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1299/2580] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:52.565979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.566274 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.566560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.568066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.587119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:52.588486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.641867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69900ba6-0d0f-4b85-8b7a-d89ef89de830 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.647298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.648077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.648369 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.648650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.649418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.651018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.660940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1282/2581] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:52.662202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1300/2582] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:52.666351 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.666681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.678085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8ffe2-e798-40d7-8628-7110fe917e11 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.681969 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55961ade-5f87-4170-afce-2dc79eb3ba17 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.683407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae44700-ce0e-4249-89ff-7a0f205aeac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.684214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae44700-ce0e-4249-89ff-7a0f205aeac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.684600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae44700-ce0e-4249-89ff-7a0f205aeac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.685018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae44700-ce0e-4249-89ff-7a0f205aeac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.685811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae44700-ce0e-4249-89ff-7a0f205aeac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.687923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae44700-ce0e-4249-89ff-7a0f205aeac5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.688044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.688970 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.689265 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.689740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] PATH_INFO: `/v3/projects/1eb1c0fb84394a80936f23e6ee57b842/users/2bc7c620527d4d5d870fd28f6c0d5a7c/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.691898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb1c0fb84394a80936f23e6ee57b842, user_id=2bc7c620527d4d5d870fd28f6c0d5a7c, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.699588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.710698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1301/2583] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:52 2022] PUT /identity/v3/projects/1eb1c0fb84394a80936f23e6ee57b842/users/2bc7c620527d4d5d870fd28f6c0d5a7c/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:52.711614 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.713958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.774904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8862d8-6e0e-4aba-927e-5d010919de42 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.778293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1283/2584] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:52.778293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.778293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.778293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.780472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.780472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.780472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.780472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.807858 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] MFA Rules not processed for user `2bc7c620527d4d5d870fd28f6c0d5a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:52.810504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.845417 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:52.850792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.852587 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.865767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1302/2585] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:52.866927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.877213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4d72e-a8da-48e6-8d61-890c0371f27e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.883404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.884657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.889204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.889204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.889204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.889819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1284/2586] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:52.889919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.898806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.900649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:52.902081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1303/2587] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:52.905903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.919732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17898f18-5d67-4506-bd3c-47d1be7e1893 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.926952 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.928208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.928208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.928460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.929297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:52.931267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:52.944074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1304/2588] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:52.949482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:52.964078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36f9a4-c51e-4f5d-855e-633fa872412f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.967460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:52.973714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.975335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.975633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.976323 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] PATH_INFO: `/v3/projects/4ef93b349cb34366a5a4925cdb35c927/users/cb8e689042f84f52abda84b606750eeb/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:52.984746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f40df-894f-452f-8c91-6c21a3823d37 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:52.990613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa15b89-89a5-489f-b3ed-97b3b9f16fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:52.991768 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa15b89-89a5-489f-b3ed-97b3b9f16fbd service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:52.992617 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa15b89-89a5-489f-b3ed-97b3b9f16fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:52.993077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa15b89-89a5-489f-b3ed-97b3b9f16fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.005759 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa15b89-89a5-489f-b3ed-97b3b9f16fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.007595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa15b89-89a5-489f-b3ed-97b3b9f16fbd service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:53.015944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef93b349cb34366a5a4925cdb35c927, user_id=cb8e689042f84f52abda84b606750eeb, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.019324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:53.028139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1285/2589] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:53.187427 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1305/2590] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:52 2022] PUT /identity/v3/projects/4ef93b349cb34366a5a4925cdb35c927/users/cb8e689042f84f52abda84b606750eeb/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:53.197981 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:53.198368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:53.198601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.282520 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] MFA Rules not processed for user `cb8e689042f84f52abda84b606750eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:53.284491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:53.366355 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1286/2591] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:53.370809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:53.409807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:53.482789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199bf4-0cb8-4ff4-ac2a-d4719d429f23 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:53.487419 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:53.488080 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:53.488528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:53.488857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.491366 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:53.492935 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138636eb-2157-4bd4-a1a4-339f456a6068 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:53.508709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:53.509650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:53.510193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:53.514013 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.515759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.519236 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:53.574375 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.575903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:53.586521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1287/2592] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:53.598297 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:53.603183 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:53.631745 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1306/2593] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:53 2022] POST /identity/v3/projects => generated 379 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:53.636103 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86318f12-60c5-4bdb-b8c7-53193d07855b service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:53.636401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:53.758017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:53.758017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:53.758017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:53.758017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] PATH_INFO: `/v3/projects/196fbe2e602e427a8d36d5a3895ccb75` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.758706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=196fbe2e602e427a8d36d5a3895ccb75)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.758706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:53.758899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891aea4-927f-4366-8fb0-87d6137403f9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:53.758899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:53.758899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:53.758899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:53.759610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:53.759610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:53.761645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.079005 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1307/2594] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:53 2022] POST /identity/v3/users => generated 296 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:54.084083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:54.087685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:54.096557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670aaf-8e27-400f-a1e1-5296eba916a0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:54.102006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '196fbe2e602e427a8d36d5a3895ccb75', 'request_id': 'req-465ef710-c409-4df4-b712-05c87e60cf3e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.167874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:54.168566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.168882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.169165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.170550 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:54.172173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.182792 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1308/2595] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:54.186939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:54.197684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2d1a7-f315-4cc1-a652-c96ecc839d43 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:54.202554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:54.203712 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.204007 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.204289 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] PATH_INFO: `/v3/projects/7c9194f8c1ca4d769b3e171277f5883b/users/69e31ccf6e834073892c3d6bf12476fb/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.223911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9194f8c1ca4d769b3e171277f5883b, user_id=69e31ccf6e834073892c3d6bf12476fb, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:54.226851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.242535 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '196fbe2e602e427a8d36d5a3895ccb75', 'request_id': 'req-465ef710-c409-4df4-b712-05c87e60cf3e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.249362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '196fbe2e602e427a8d36d5a3895ccb75', 'request_id': 'req-465ef710-c409-4df4-b712-05c87e60cf3e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.264206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] The token cache is being invalidate because project 196fbe2e602e427a8d36d5a3895ccb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:54.264500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.265219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.267088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1288/2596] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:53 2022] DELETE /identity/v3/projects/196fbe2e602e427a8d36d5a3895ccb75 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:54.271573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:54.272875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:54.289588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ef710-c409-4df4-b712-05c87e60cf3e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:54.295451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:54.296627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.296951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.297390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] PATH_INFO: `/v3/users/8ad5d804816942f9b33d2eac8085f255` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.298704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad5d804816942f9b33d2eac8085f255)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:54.301293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.354515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1309/2597] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:54 2022] PUT /identity/v3/projects/7c9194f8c1ca4d769b3e171277f5883b/users/69e31ccf6e834073892c3d6bf12476fb/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:54.358432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:54.360029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:54.427472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fee07f-b2aa-42e7-ade7-be12ba2c07b8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:54.438557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:54.439428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.439906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.440204 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.440989 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:54.442608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.453072 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1310/2598] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:54.457396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:54.466938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e57aa2-c012-47ba-9ff4-8bdb36a3073c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:54.472217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e837e-757f-47f6-81a2-40c6b204f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:54.473180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e837e-757f-47f6-81a2-40c6b204f68d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.473477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e837e-757f-47f6-81a2-40c6b204f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.473785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e837e-757f-47f6-81a2-40c6b204f68d admin admin] PATH_INFO: `/v3/projects/7c9194f8c1ca4d769b3e171277f5883b/users/69e31ccf6e834073892c3d6bf12476fb/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.475753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e837e-757f-47f6-81a2-40c6b204f68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9194f8c1ca4d769b3e171277f5883b, user_id=69e31ccf6e834073892c3d6bf12476fb, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:54.478481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e837e-757f-47f6-81a2-40c6b204f68d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.491045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1311/2599] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:54 2022] PUT /identity/v3/projects/7c9194f8c1ca4d769b3e171277f5883b/users/69e31ccf6e834073892c3d6bf12476fb/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:54.502381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.502819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.503107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.550807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] MFA Rules not processed for user `69e31ccf6e834073892c3d6bf12476fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:54.550807 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:54.585310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad5d804816942f9b33d2eac8085f255'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.612839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad5d804816942f9b33d2eac8085f255'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.627369 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad5d804816942f9b33d2eac8085f255'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.657605 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1312/2600] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:54.671033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:54.672555 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:54.751292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcef3d-2fa9-4611-bd7c-971f133f230e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:54.755891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:54.756896 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:54.757578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:54.758116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:54.768323 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad5d804816942f9b33d2eac8085f255'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.769240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad5d804816942f9b33d2eac8085f255'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:54.771035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:54.773162 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:54.777611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1289/2601] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:54 2022] DELETE /identity/v3/users/8ad5d804816942f9b33d2eac8085f255 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:54.797477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1313/2602] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:54.849042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:55.004649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:55.006215 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:55.037636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:55.087938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c92f3-7df7-45f1-a534-b9e47f1cdd56 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:55.092732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:55.093652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.096619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.096619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.097770 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:55.169128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:55.174036 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665ae3a-3d3f-4dbd-b42a-f66e860fb8dd service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:55.180134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1290/2603] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:55.266780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:55.333951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49e0da-cf18-408a-a78b-d10b58f028d9 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:55.396408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:55.397309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.397629 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.397955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] PATH_INFO: `/v3/projects/a4e0106b85cc4d06aa9967bcae73e9f7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.399094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e0106b85cc4d06aa9967bcae73e9f7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:55.401080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:55.550626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e0106b85cc4d06aa9967bcae73e9f7', 'request_id': 'req-f7966763-69b5-4dd3-a12a-ce02d17823bc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:55.556865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e0106b85cc4d06aa9967bcae73e9f7', 'request_id': 'req-f7966763-69b5-4dd3-a12a-ce02d17823bc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:55.719886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e0106b85cc4d06aa9967bcae73e9f7', 'request_id': 'req-f7966763-69b5-4dd3-a12a-ce02d17823bc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:55.731634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] The token cache is being invalidate because project a4e0106b85cc4d06aa9967bcae73e9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:55.731922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:55.732545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:55.734117 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1314/2604] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:55 2022] DELETE /identity/v3/projects/a4e0106b85cc4d06aa9967bcae73e9f7 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:55.739891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b3b50-e543-4781-a580-92ecc7d7e808 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.740178 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b3b50-e543-4781-a580-92ecc7d7e808 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.740481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b3b50-e543-4781-a580-92ecc7d7e808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.759984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4c1b3b50-e543-4781-a580-92ecc7d7e808 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:55.761441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c1b3b50-e543-4781-a580-92ecc7d7e808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:55.807459 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:55.808964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:55.834415 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1291/2605] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:55.842978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.843294 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.846823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.863656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:55.864950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:55.873480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1292/2606] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:55.877379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:55.886678 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9899ef-bbb0-47ae-a18c-788310dec9ab None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:55.887025 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966763-69b5-4dd3-a12a-ce02d17823bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:55.891513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:55.891836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:55.892110 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.892562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.892796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.892914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.893085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.893365 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] PATH_INFO: `/v3/users/cb8e689042f84f52abda84b606750eeb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.893665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:55.893980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:55.894590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb8e689042f84f52abda84b606750eeb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:55.895560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:55.896499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:55.916971 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1293/2607] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:50:55.918518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:55.930268 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4c443-0f59-4d34-83af-6249d908cae0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:55.936461 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:55.937327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:55.937685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:55.938190 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:55.938994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:55.941404 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.050764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:56.056372 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1294/2608] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:55 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:56.060539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.120891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f157-9ae5-4278-a1c6-d49a26ca6840 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:56.125616 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:56.130842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.130842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.130842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.130842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:56.130842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.203753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb8e689042f84f52abda84b606750eeb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:56.211161 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb8e689042f84f52abda84b606750eeb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:56.322333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1295/2609] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:56 2022] POST /identity/v3/users => generated 295 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:56.326283 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.337025 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e1a87-7042-4ffd-8367-a56462a8fbd0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:56.341843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:56.342494 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.342811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.343075 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.343867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:56.346272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.356284 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1296/2610] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:56.360105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.367844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb8e689042f84f52abda84b606750eeb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:56.368547 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb8e689042f84f52abda84b606750eeb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:56.369284 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-22081269-96b3-4ce4-9dc5-46da86d80540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb8e689042f84f52abda84b606750eeb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:56.370266 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08706c15-498e-444d-885b-ed565995ee9e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:56.377138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:56.377138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.377138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.377138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.377903 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:56.379191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.391776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1315/2611] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:55 2022] DELETE /identity/v3/users/cb8e689042f84f52abda84b606750eeb => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:56.397253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.397594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1297/2612] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:56.400840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.401252 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.401970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.403679 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:56.417460 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca04026-9217-4023-a2d7-20e7ffd1b5e0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:56.421772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:56.423040 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:56.478845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:56.479898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.488811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.489253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] PATH_INFO: `/v3/projects/4661e25bd47948f195f76a9783eb22b3/users/e5d3ff7f152144f9b3ce26271911164f/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.496988 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1316/2613] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:56.509012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.510672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:56.520528 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4661e25bd47948f195f76a9783eb22b3, user_id=e5d3ff7f152144f9b3ce26271911164f, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:56.523421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.567974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:56.571250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b2bc-4ae6-4ee4-b122-b0d39387e66a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:56.574931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd40264-eea2-4928-8a16-430608bc5323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:56.575608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd40264-eea2-4928-8a16-430608bc5323 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.575946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd40264-eea2-4928-8a16-430608bc5323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.576225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd40264-eea2-4928-8a16-430608bc5323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.586913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd40264-eea2-4928-8a16-430608bc5323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:56.588340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd40264-eea2-4928-8a16-430608bc5323 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.597937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1317/2614] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:56.625465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1298/2615] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:56 2022] PUT /identity/v3/projects/4661e25bd47948f195f76a9783eb22b3/users/e5d3ff7f152144f9b3ce26271911164f/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:56.637102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.637390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.637601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.665625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] MFA Rules not processed for user `e5d3ff7f152144f9b3ce26271911164f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:56.668425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:56.733447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1318/2616] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:56.738069 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.801692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f384d-38da-4e49-9232-578cad8ddcfb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:56.807716 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:56.808392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:56.808705 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:56.808989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:56.809718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:56.811599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:56.964256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1299/2617] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:56 2022] POST /identity/v3/projects => generated 389 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:50:56.969198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:56.979711 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3b5e3-c8f7-4809-8afa-602e25851266 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.037922 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.038598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.038886 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.039165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.039853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.041833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.100907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.102890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:57.115926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ee67-aa99-438e-a0b1-30be5a7c0d4d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.122654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.123147 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.123281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.123594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] PATH_INFO: `/v3/projects/4ef93b349cb34366a5a4925cdb35c927` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.125571 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef93b349cb34366a5a4925cdb35c927)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.128187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.172267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1319/2618] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:56 2022] POST /identity/v3/users => generated 301 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:57.178778 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.189386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd314a1-74d2-4233-9bc3-529e171e9f20 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.194181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.194849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.195142 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.195425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.196185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.197823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.208823 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1320/2619] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:50:57.212786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.223466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e646-30a5-4f43-b361-082ef85ef3db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.228138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.229079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.229359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.229636 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] PATH_INFO: `/v3/projects/55e5772c2df24c6f95bb08b55c5c2af2/users/48be34b348744aa19d06454f974ab504/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.243906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e5772c2df24c6f95bb08b55c5c2af2, user_id=48be34b348744aa19d06454f974ab504, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.246566 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.360645 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1321/2620] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:57 2022] PUT /identity/v3/projects/55e5772c2df24c6f95bb08b55c5c2af2/users/48be34b348744aa19d06454f974ab504/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:57.365171 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.369158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef93b349cb34366a5a4925cdb35c927', 'request_id': 'req-de2cae92-a352-4852-8378-ff55ae607470'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.426207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9bdee-b45f-4abd-8c66-dcfb9c19c9be admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.462656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef93b349cb34366a5a4925cdb35c927', 'request_id': 'req-de2cae92-a352-4852-8378-ff55ae607470'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.468866 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef93b349cb34366a5a4925cdb35c927', 'request_id': 'req-de2cae92-a352-4852-8378-ff55ae607470'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.481096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.481773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.482058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.482318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] The token cache is being invalidate because project 4ef93b349cb34366a5a4925cdb35c927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:57.482428 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.482674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.483239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.483368 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.484860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.484994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1300/2621] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:57 2022] DELETE /identity/v3/projects/4ef93b349cb34366a5a4925cdb35c927 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:57.489265 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.490652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:57.498679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1322/2622] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:57.499475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.501127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:57.503457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cae92-a352-4852-8378-ff55ae607470 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.508654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.509556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.509978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.510270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] PATH_INFO: `/v3/users/69e31ccf6e834073892c3d6bf12476fb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.511489 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e31ccf6e834073892c3d6bf12476fb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.513457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.516481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfc4a6-ff39-4f6c-95d8-8d2a95229f96 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.522113 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04530a6-4b0e-45d8-a096-4919c236300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.522994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04530a6-4b0e-45d8-a096-4919c236300c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.523282 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04530a6-4b0e-45d8-a096-4919c236300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.523560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04530a6-4b0e-45d8-a096-4919c236300c admin admin] PATH_INFO: `/v3/projects/55e5772c2df24c6f95bb08b55c5c2af2/users/48be34b348744aa19d06454f974ab504/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.525672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04530a6-4b0e-45d8-a096-4919c236300c admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e5772c2df24c6f95bb08b55c5c2af2, user_id=48be34b348744aa19d06454f974ab504, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.528366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04530a6-4b0e-45d8-a096-4919c236300c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.540403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1323/2623] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:57 2022] PUT /identity/v3/projects/55e5772c2df24c6f95bb08b55c5c2af2/users/48be34b348744aa19d06454f974ab504/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:57.551777 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.552075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.552359 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.591865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] MFA Rules not processed for user `48be34b348744aa19d06454f974ab504`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:57.591865 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:57.685999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1324/2624] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:57.697057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.698891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:57.700062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:57.756781 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631b7c0-5a5a-4200-82a7-529ff2df3c29 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:57.760519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:57.761199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:57.761491 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:57.761758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:57.771961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:57.778919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:57.782992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1325/2625] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:57.883720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e31ccf6e834073892c3d6bf12476fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.907058 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e31ccf6e834073892c3d6bf12476fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.915090 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:57.917605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e31ccf6e834073892c3d6bf12476fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.996300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e31ccf6e834073892c3d6bf12476fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.996989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8064269e-0ed4-45fa-b6ba-a87aa72744e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e31ccf6e834073892c3d6bf12476fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:57.998057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af3456-f6ab-4c75-962d-26821b910d90 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:58.004255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:58.005172 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.005674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1301/2626] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:57 2022] DELETE /identity/v3/users/69e31ccf6e834073892c3d6bf12476fb => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:58.005810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.005810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] PATH_INFO: `/v3/users/e5d3ff7f152144f9b3ce26271911164f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.007129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d3ff7f152144f9b3ce26271911164f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:58.009518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:58.216208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d3ff7f152144f9b3ce26271911164f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.224940 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d3ff7f152144f9b3ce26271911164f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.285923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:58.310795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d3ff7f152144f9b3ce26271911164f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.311479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d3ff7f152144f9b3ce26271911164f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.312390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d3ff7f152144f9b3ce26271911164f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.334101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1326/2627] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:57 2022] DELETE /identity/v3/users/e5d3ff7f152144f9b3ce26271911164f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:58.339706 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.340904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.341414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.390958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:58.392750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:58.449078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1302/2628] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:58.460005 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:58.460179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:58.461609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:58.462248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:58.479796 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc07f-29d4-4458-ac9b-2243ce42fb76 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:58.485226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:58.486099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.486472 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.486849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] PATH_INFO: `/v3/projects/7c9194f8c1ca4d769b3e171277f5883b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.488045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c9194f8c1ca4d769b3e171277f5883b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:58.493515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:58.519053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:58.539348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff296f2-9483-4ec6-ae22-7ec7e2a790b1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:58.542912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88955c7e-1a8f-40b1-b21a-a4e62acff6ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:58.556350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88955c7e-1a8f-40b1-b21a-a4e62acff6ba service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.556350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88955c7e-1a8f-40b1-b21a-a4e62acff6ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.556350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88955c7e-1a8f-40b1-b21a-a4e62acff6ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.556350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88955c7e-1a8f-40b1-b21a-a4e62acff6ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:58.556350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88955c7e-1a8f-40b1-b21a-a4e62acff6ba service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:58.563987 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1303/2629] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:50:58.592784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c9194f8c1ca4d769b3e171277f5883b', 'request_id': 'req-2ef47f4d-2a11-4e28-854a-681c6a87e518'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.601756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c9194f8c1ca4d769b3e171277f5883b', 'request_id': 'req-2ef47f4d-2a11-4e28-854a-681c6a87e518'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.744525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c9194f8c1ca4d769b3e171277f5883b', 'request_id': 'req-2ef47f4d-2a11-4e28-854a-681c6a87e518'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.756149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] The token cache is being invalidate because project 7c9194f8c1ca4d769b3e171277f5883b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:58.756474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.757153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ef47f4d-2a11-4e28-854a-681c6a87e518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:58.758765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1327/2630] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:58 2022] DELETE /identity/v3/projects/7c9194f8c1ca4d769b3e171277f5883b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:58.763750 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.764043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.764347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.816433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:58.817703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:58.882898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1304/2631] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:58.890634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.890928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.891212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.910134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:58.911342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:58.925094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1328/2632] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:50:58.929347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:58.939095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa4d79-3592-43a0-9579-dd36770f805a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:58.943867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:58.944654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.944955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.945260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.946009 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:50:58.946491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:58.948134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:58.960292 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1305/2633] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:50:58.964717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:58.974782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef9d72-eeca-4fec-85f5-d0c1b79b737d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:58.979322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:58.979467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:58.980100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:58.980383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:58.980660 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:58.981366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:58.983292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.086046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:59.092553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.094794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:59.101598 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1329/2634] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:50:58 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:59.103394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.108901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a66570-f8a8-4d14-bea9-e8014ca89e7c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.167579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.168450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.169354 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.169643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] PATH_INFO: `/v3/projects/4661e25bd47948f195f76a9783eb22b3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.170842 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4661e25bd47948f195f76a9783eb22b3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.172854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.177030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22c99b-bb45-453e-90aa-2a083a39d193 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.181930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.185096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.185096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.185096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.185096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.185838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.276923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1330/2635] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:50:59 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:50:59.280933 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.290244 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4661e25bd47948f195f76a9783eb22b3', 'request_id': 'req-478f938d-edae-4de8-9d63-67a49aa05c19'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.291371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8721241-95a8-4ad5-98dd-81bbc3b9f370 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.346225 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.346815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.347101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.347382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.348106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.349729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.356269 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:59.361149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1331/2636] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:59.365066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.374420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a6fd6-2fd9-4a93-aa16-d632f844a185 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.379304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.379953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.380245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.380530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.381288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.382873 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.393600 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1332/2637] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:50:59.397790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.399925 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4661e25bd47948f195f76a9783eb22b3', 'request_id': 'req-478f938d-edae-4de8-9d63-67a49aa05c19'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.408851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715355c9-1485-44cf-a9c4-72d9e3b3f90c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.412852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4661e25bd47948f195f76a9783eb22b3', 'request_id': 'req-478f938d-edae-4de8-9d63-67a49aa05c19'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.414043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f035f-665e-440d-abe7-afc6d75ad6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.415659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f035f-665e-440d-abe7-afc6d75ad6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.415946 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f035f-665e-440d-abe7-afc6d75ad6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.416228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f035f-665e-440d-abe7-afc6d75ad6c3 admin admin] PATH_INFO: `/v3/projects/dfd25b0deafe47e0ad3f7c729d6c5d76/users/4517f235551f44d4852c262fc1297882/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.425197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] The token cache is being invalidate because project 4661e25bd47948f195f76a9783eb22b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:50:59.425487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.426180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.427747 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1306/2638] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:50:59 2022] DELETE /identity/v3/projects/4661e25bd47948f195f76a9783eb22b3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:59.431746 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.433939 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:59.434075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f035f-665e-440d-abe7-afc6d75ad6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd25b0deafe47e0ad3f7c729d6c5d76, user_id=4517f235551f44d4852c262fc1297882, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.435904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f035f-665e-440d-abe7-afc6d75ad6c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.446393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f938d-edae-4de8-9d63-67a49aa05c19 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.450970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.451840 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.452123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.452402 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] PATH_INFO: `/v3/users/48be34b348744aa19d06454f974ab504` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.453891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=48be34b348744aa19d06454f974ab504)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.457079 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.522979 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1333/2639] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:50:59 2022] PUT /identity/v3/projects/dfd25b0deafe47e0ad3f7c729d6c5d76/users/4517f235551f44d4852c262fc1297882/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:50:59.535008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.535299 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.535578 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.566687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] MFA Rules not processed for user `4517f235551f44d4852c262fc1297882`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:50:59.568107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:59.637422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1334/2640] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:50:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:50:59.648855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:50:59.650169 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:50:59.667027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48be34b348744aa19d06454f974ab504'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.669929 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:59.680950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48be34b348744aa19d06454f974ab504'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.687678 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48be34b348744aa19d06454f974ab504'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.707845 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0812cf1-e32c-42b9-bb72-7fe53aff8144 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:50:59.711704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:50:59.712557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:50:59.712882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:50:59.713250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:50:59.722349 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:59.724278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:50:59.725803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:50:59.734826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1335/2641] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:50:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:50:59.744553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48be34b348744aa19d06454f974ab504'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.745236 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-56a4bc6e-31db-4be9-859e-77603d5d06dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48be34b348744aa19d06454f974ab504'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:50:59.753748 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1307/2642] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:50:59 2022] DELETE /identity/v3/users/48be34b348744aa19d06454f974ab504 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:50:59.760198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:50:59.785634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:00.156249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:00.219942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:00.300267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6451a-4f05-4e6d-9e48-72c14714fc20 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:00.355092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:00.359511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:00.359511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:00.359511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] PATH_INFO: `/v3/projects/55e5772c2df24c6f95bb08b55c5c2af2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:00.359511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e5772c2df24c6f95bb08b55c5c2af2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:00.359876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:00.497641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e5772c2df24c6f95bb08b55c5c2af2', 'request_id': 'req-e7a4d031-3d2e-4142-8c7e-dc9af257941e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:00.503747 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e5772c2df24c6f95bb08b55c5c2af2', 'request_id': 'req-e7a4d031-3d2e-4142-8c7e-dc9af257941e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:00.605378 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e5772c2df24c6f95bb08b55c5c2af2', 'request_id': 'req-e7a4d031-3d2e-4142-8c7e-dc9af257941e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:00.617185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] The token cache is being invalidate because project 55e5772c2df24c6f95bb08b55c5c2af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:00.617486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:00.618133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a4d031-3d2e-4142-8c7e-dc9af257941e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:00.620057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1336/2643] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:00 2022] DELETE /identity/v3/projects/55e5772c2df24c6f95bb08b55c5c2af2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:00.632576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:00.633088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:00.633173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:00.652138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:00.653409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:00.706140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:00.721431 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1308/2644] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:00.730148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:00.730445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:00.730740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:00.749440 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:00.750646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:00.758966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1337/2645] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:00.763165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:00.774867 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0679-154d-4aff-a0b0-d50b9646c61e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:00.779873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:00.780551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:00.780865 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:00.781146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:00.781935 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:00.782272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:00.784919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:00.797655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1309/2646] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:51:00.802319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:00.811678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940203b9-1e44-4e1c-ba6d-ff2e67c6a337 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:00.816626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:00.817302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:00.817644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:00.817926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:00.818634 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:00.820521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:00.968860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1338/2647] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:00 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:00.973595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:00.983600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ec799-3a8e-4098-bca7-8546e44d1de8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.043320 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.043996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.044286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.045711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.045711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.047333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.162172 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1310/2648] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:00 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:01.166471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.178553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb7f8-831f-4463-83f3-c79875a5e15e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.183607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.184283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.184573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.184882 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.185625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.187287 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.199184 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1339/2649] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:01.203593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.215634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aee71-851d-4c7d-9494-cb964c098786 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.220513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ccab0-0826-452f-b28b-9354a9ad3b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.221181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ccab0-0826-452f-b28b-9354a9ad3b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.221471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ccab0-0826-452f-b28b-9354a9ad3b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.221756 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ccab0-0826-452f-b28b-9354a9ad3b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.222491 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ccab0-0826-452f-b28b-9354a9ad3b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.224132 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ccab0-0826-452f-b28b-9354a9ad3b70 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.234977 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1311/2650] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:01.239759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.250437 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722fddd-6f6b-4744-a914-760671f7a447 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.256081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.256998 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.261361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.261361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] PATH_INFO: `/v3/projects/2eea067abacd47f58056b6b0e7f32a93/users/b4590cfa29bf4a9bb6c271f72964628b/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.275834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eea067abacd47f58056b6b0e7f32a93, user_id=b4590cfa29bf4a9bb6c271f72964628b, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.281063 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.377859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1340/2651] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:01 2022] PUT /identity/v3/projects/2eea067abacd47f58056b6b0e7f32a93/users/b4590cfa29bf4a9bb6c271f72964628b/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:01.388242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.388537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.388849 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.416030 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] MFA Rules not processed for user `b4590cfa29bf4a9bb6c271f72964628b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:01.417325 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:01.451612 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:01.483559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1312/2652] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:01.488213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.549868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd421-8602-485e-b2a3-69e8402d68a1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.554536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.555196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.555486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.555770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.556471 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.558583 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.664963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1341/2653] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:01 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:01.669157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.679832 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a652485-5d6e-47c0-a94f-f31f1b13740e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.736648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.737267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.737574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.737838 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.738531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.740443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.871794 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1313/2654] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:01 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:01.874814 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.885215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23018d-3a43-4ffa-90cc-be068e9c1450 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.892834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.892834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.892834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.892834 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.893406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.894080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:01.906432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1342/2655] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:01.913994 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:01.924874 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642365a-9bd5-49e4-8243-155103d0cab3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:01.929596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:01.930476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:01.930764 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:01.931042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] PATH_INFO: `/v3/projects/6f9f24d7da3440bea2a1e5fe23622081/users/36528acd7d4149b496c7ff25d1d5a897/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:01.946295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f9f24d7da3440bea2a1e5fe23622081, user_id=36528acd7d4149b496c7ff25d1d5a897, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:01.949011 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:02.052621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1314/2656] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:01 2022] PUT /identity/v3/projects/6f9f24d7da3440bea2a1e5fe23622081/users/36528acd7d4149b496c7ff25d1d5a897/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:02.056761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:02.066913 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5f22-e1fd-46ab-90cc-9de31e37d7b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:02.125377 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884cd6b-f9ae-44bc-9ad3-4c7355090c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:02.126084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884cd6b-f9ae-44bc-9ad3-4c7355090c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:02.126513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884cd6b-f9ae-44bc-9ad3-4c7355090c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:02.126789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884cd6b-f9ae-44bc-9ad3-4c7355090c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:02.127525 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884cd6b-f9ae-44bc-9ad3-4c7355090c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:02.129580 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884cd6b-f9ae-44bc-9ad3-4c7355090c2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:02.139776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1343/2657] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:02.144513 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:02.155158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8813531-6d9d-40e5-8a02-99aa1f50d945 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:02.161161 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:02.162043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:02.162332 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:02.162613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] PATH_INFO: `/v3/projects/6f9f24d7da3440bea2a1e5fe23622081/users/36528acd7d4149b496c7ff25d1d5a897/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:02.164590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f9f24d7da3440bea2a1e5fe23622081, user_id=36528acd7d4149b496c7ff25d1d5a897, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:02.167453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:02.179722 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1315/2658] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:02 2022] PUT /identity/v3/projects/6f9f24d7da3440bea2a1e5fe23622081/users/36528acd7d4149b496c7ff25d1d5a897/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:02.191419 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:02.191915 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:02.192015 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:02.205642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:02.219963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] MFA Rules not processed for user `36528acd7d4149b496c7ff25d1d5a897`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:02.222096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:02.292124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1344/2659] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:02.303992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:02.305781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:02.372270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a533a3-e76d-427c-8433-5503cdfb6e73 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:02.375951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:02.377242 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:02.377530 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:02.377809 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:02.423944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:02.440557 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:02.442053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:02.455386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1316/2660] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:07.920410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:07.931392 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f633b-e36e-4736-b17a-913b1fddf7c1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:07.935401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:07.936043 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:07.936333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:07.936616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:07.946623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:07.952541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:07.958387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1345/2661] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:09.195852 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:09.197324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:09.264652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe773b-437d-4cfc-839c-8cab932a4e0c service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:09.275181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:09.275843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:09.276138 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:09.276421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:09.278421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:09.342279 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:09.343719 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea0d27-024f-4ab3-9223-14837fc9ad73 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:09.354341 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1317/2662] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:14.270361 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:14.340346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76cbd-2159-4050-b230-de1b8c00f1da service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:14.345848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:14.346737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:14.347022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:14.347294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] PATH_INFO: `/v3/users/b4590cfa29bf4a9bb6c271f72964628b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:14.348546 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4590cfa29bf4a9bb6c271f72964628b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:14.350721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:14.556273 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4590cfa29bf4a9bb6c271f72964628b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:14.565008 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4590cfa29bf4a9bb6c271f72964628b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:14.670202 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4590cfa29bf4a9bb6c271f72964628b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:14.671235 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4590cfa29bf4a9bb6c271f72964628b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:14.671529 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4590cfa29bf4a9bb6c271f72964628b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:14.688891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:14.697465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1346/2663] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:14 2022] DELETE /identity/v3/users/b4590cfa29bf4a9bb6c271f72964628b => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:14.701137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:14.701649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:14.701751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:14.727970 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:14.729228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:14.815805 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1318/2664] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:14.823553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:14.825134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:14.909585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39475fd7-30b2-4a22-8595-318cdb88c57b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:14.919026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:14.920370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:14.920875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:14.921312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:14.936893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:14.940707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:14.951957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1347/2665] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:15.417471 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:15.418943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:15.433783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a21363-3912-4ff2-9f38-113f8a5d8904 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:15.438567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:15.439447 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:15.443972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:15.443972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] PATH_INFO: `/v3/projects/2eea067abacd47f58056b6b0e7f32a93` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:15.443972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eea067abacd47f58056b6b0e7f32a93)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:15.443972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:15.584621 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:15.600270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eea067abacd47f58056b6b0e7f32a93', 'request_id': 'req-82f70458-05c8-481f-b939-04a5b22dc7e7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:15.718955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eea067abacd47f58056b6b0e7f32a93', 'request_id': 'req-82f70458-05c8-481f-b939-04a5b22dc7e7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:15.725472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eea067abacd47f58056b6b0e7f32a93', 'request_id': 'req-82f70458-05c8-481f-b939-04a5b22dc7e7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:15.736906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] The token cache is being invalidate because project 2eea067abacd47f58056b6b0e7f32a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:15.737202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:15.737851 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:15.739398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1319/2666] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:15 2022] DELETE /identity/v3/projects/2eea067abacd47f58056b6b0e7f32a93 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:15.743640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:15.753771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330467-856a-406c-b0aa-1d677ec12210 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:15.810254 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:15.811147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:15.811429 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:15.811707 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] PATH_INFO: `/v3/users/36528acd7d4149b496c7ff25d1d5a897` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:15.813022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36528acd7d4149b496c7ff25d1d5a897)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:15.815021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:16.078866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36528acd7d4149b496c7ff25d1d5a897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:16.086678 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36528acd7d4149b496c7ff25d1d5a897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:16.177977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36528acd7d4149b496c7ff25d1d5a897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:16.178603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36528acd7d4149b496c7ff25d1d5a897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:16.179217 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dabe6e0-7776-46a9-9d0d-d47dfd08a839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36528acd7d4149b496c7ff25d1d5a897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:16.197949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1348/2667] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:15 2022] DELETE /identity/v3/users/36528acd7d4149b496c7ff25d1d5a897 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:16.675540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:16.677051 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:16.691854 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f70458-05c8-481f-b939-04a5b22dc7e7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:16.751951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:16.752895 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:16.753179 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:16.753452 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] PATH_INFO: `/v3/projects/6f9f24d7da3440bea2a1e5fe23622081` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:16.754672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f9f24d7da3440bea2a1e5fe23622081)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:16.756561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:16.938896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f9f24d7da3440bea2a1e5fe23622081', 'request_id': 'req-1f38c51c-64b7-4a3d-87f8-753b090a3768'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:17.006775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f9f24d7da3440bea2a1e5fe23622081', 'request_id': 'req-1f38c51c-64b7-4a3d-87f8-753b090a3768'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:17.013710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f9f24d7da3440bea2a1e5fe23622081', 'request_id': 'req-1f38c51c-64b7-4a3d-87f8-753b090a3768'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:17.036406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] The token cache is being invalidate because project 6f9f24d7da3440bea2a1e5fe23622081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:17.036736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:17.037406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1f38c51c-64b7-4a3d-87f8-753b090a3768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:17.038953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1320/2668] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:16 2022] DELETE /identity/v3/projects/6f9f24d7da3440bea2a1e5fe23622081 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:17.051239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.051864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.051864 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.073623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:17.074849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:17.090099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:17.153849 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1349/2669] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:17.164310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.164610 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.164930 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.187552 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:17.188850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:17.197335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1321/2670] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:17.201725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.212657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab38a93-cb28-4e50-bcd3-798c7d00475f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.217422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.218081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.218370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.218766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.219513 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:17.220431 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.222101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.234058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1350/2671] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:51:17.240286 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.250510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b951be-4ccb-4180-9c2c-87502fa18303 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.255664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.256338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.256664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.257048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.257789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.259688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.323898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.325642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:17.420628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a3596-eac7-41a3-9baa-2d644f6dda70 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.425909 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.426559 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.426853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.427136 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.428966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:17.440124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.441749 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.452016 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1351/2672] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:51:17.456698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1322/2673] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:17 2022] POST /identity/v3/projects => generated 359 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:17.461785 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.472624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12bb8a-a8b1-46e0-af41-02c7f99224fd service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.530919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.531646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.531957 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.532247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.533018 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.534936 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.653200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1352/2674] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:17 2022] POST /identity/v3/users => generated 280 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:17.657551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.667816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bd9e8-e531-43fc-bca7-2093adf72fa7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.672291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.672986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.673288 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.673550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.674272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.675864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.686409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1323/2675] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:17.691201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.701596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433ee1-d4cf-4937-9242-bffa089b0fae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.707644 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3d622-40d0-4d92-b8c7-b7d4c16844a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.708327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3d622-40d0-4d92-b8c7-b7d4c16844a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.708627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3d622-40d0-4d92-b8c7-b7d4c16844a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.708943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3d622-40d0-4d92-b8c7-b7d4c16844a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.709708 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3d622-40d0-4d92-b8c7-b7d4c16844a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.711394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3d622-40d0-4d92-b8c7-b7d4c16844a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.712828 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:17.722324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1353/2676] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:17.726609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.736752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0863-f16f-4540-871d-4d53b54ced8d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:17.741427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:17.742306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.742591 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.742898 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] PATH_INFO: `/v3/projects/a932a0169b384229a644111bcfd3f9af/users/d1bade90fa7943349cbfff6da96a3b5e/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.745299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:17.758697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a932a0169b384229a644111bcfd3f9af, user_id=d1bade90fa7943349cbfff6da96a3b5e, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:17.761439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:17.839511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1324/2677] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:17 2022] PUT /identity/v3/projects/a932a0169b384229a644111bcfd3f9af/users/d1bade90fa7943349cbfff6da96a3b5e/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:17.852002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:17.852300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:17.852588 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:17.879851 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] MFA Rules not processed for user `d1bade90fa7943349cbfff6da96a3b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:17.881035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:17.952124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1354/2678] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:17 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:17.982451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:17.984228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:18.047979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dbb2a-ce14-4bb2-8b78-70ed4b3c0cbb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:18.053421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de180ed0-4773-4f0a-ab3e-d80fa9010d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:18.053421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de180ed0-4773-4f0a-ab3e-d80fa9010d8a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:18.053421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de180ed0-4773-4f0a-ab3e-d80fa9010d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:18.053421 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de180ed0-4773-4f0a-ab3e-d80fa9010d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:18.063349 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de180ed0-4773-4f0a-ab3e-d80fa9010d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:18.064803 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de180ed0-4773-4f0a-ab3e-d80fa9010d8a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:18.075886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1325/2679] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:17 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:18.094279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:18.170185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c432cd-6c7f-488c-89ef-202b3e14b495 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:18.175130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:18.176096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:18.176391 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:18.176694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] PATH_INFO: `/v3/users/d1bade90fa7943349cbfff6da96a3b5e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:18.178100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bade90fa7943349cbfff6da96a3b5e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:18.181630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:18.421484 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bade90fa7943349cbfff6da96a3b5e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:18.430176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bade90fa7943349cbfff6da96a3b5e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:18.528208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bade90fa7943349cbfff6da96a3b5e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:18.536557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bade90fa7943349cbfff6da96a3b5e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:18.536557 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bade90fa7943349cbfff6da96a3b5e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:18.573876 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1355/2680] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:18 2022] DELETE /identity/v3/users/d1bade90fa7943349cbfff6da96a3b5e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:18.579632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:18.579932 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:18.580210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:18.601815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:18.603095 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:18.682443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1326/2681] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:18.694482 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:18.696051 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:18.702812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:18.795397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342e62f-f523-440d-a98a-b73e986b82db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:18.800735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e12c68-0f55-4410-a7a8-3d351ec166b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:18.801618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12c68-0f55-4410-a7a8-3d351ec166b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:18.802209 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12c68-0f55-4410-a7a8-3d351ec166b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:18.802517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12c68-0f55-4410-a7a8-3d351ec166b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:18.814976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12c68-0f55-4410-a7a8-3d351ec166b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:18.816310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12c68-0f55-4410-a7a8-3d351ec166b6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:18.830266 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1356/2682] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:19.317380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:19.318894 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:19.331916 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59c0cb-7c00-4c5c-b1a2-d755ef17b66e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:19.336529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:19.337476 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:19.337752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:19.338031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] PATH_INFO: `/v3/projects/a932a0169b384229a644111bcfd3f9af` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:19.339468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a932a0169b384229a644111bcfd3f9af)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:19.341407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:19.486671 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a932a0169b384229a644111bcfd3f9af', 'request_id': 'req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:19.589680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a932a0169b384229a644111bcfd3f9af', 'request_id': 'req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:19.596902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a932a0169b384229a644111bcfd3f9af', 'request_id': 'req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:19.609924 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] The token cache is being invalidate because project a932a0169b384229a644111bcfd3f9af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:19.610253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:19.610982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6399f5fc-4fcc-4c50-b9ff-a8de57cb46ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:19.612830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1327/2683] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:19 2022] DELETE /identity/v3/projects/a932a0169b384229a644111bcfd3f9af => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:19.618186 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:19.618651 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:19.618771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:19.636803 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:19.638037 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:19.699898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1357/2684] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:19.709901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:19.710191 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:19.710470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:19.729870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:19.731090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:19.739834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1328/2685] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:19.744108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:19.755064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6669f3-e9ba-4953-9381-4b6cae72c34b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:19.760384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:19.761386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:19.761386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:19.761639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:19.762380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:19.762726 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:19.764341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:19.778334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1358/2686] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:51:19.786885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:19.791363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:19.796963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393935-bfe3-46c1-9e21-8978f71edc87 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:19.806022 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:19.806698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:19.806984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:19.807260 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:19.807959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:19.809886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:19.865277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:19.867031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:19.881723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48864c1-b805-4165-b032-4660000fa045 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:19.888371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:19.888737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:19.889085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:19.889422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] PATH_INFO: `/v3/users/4517f235551f44d4852c262fc1297882` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:19.890655 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4517f235551f44d4852c262fc1297882)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:19.892767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:19.910580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1329/2687] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:19 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:19.914692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:19.999435 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed6480-cfa9-4f11-a9f4-34b509f2fb2d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.018938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.019277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.019511 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.019785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.020480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.022632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.169720 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1330/2688] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:19 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:51:20.174654 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:20.178725 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4517f235551f44d4852c262fc1297882'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:20.185385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b555-f86c-4eb7-9ba6-562d910b138f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.186130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4517f235551f44d4852c262fc1297882'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:20.190769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.191432 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.191714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.192019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.192785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.194376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.204587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1331/2689] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:20.208463 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:20.217843 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76c5f4-0e11-43e8-be37-42e8e3cd1149 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.222470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.223074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.223359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.223636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.224337 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.225923 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.236901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1332/2690] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:51:20.240853 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:20.254454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270c437-01c9-443b-9a01-a58686e071d8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.256209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712014-c495-4027-a8b5-6544ac2648bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.257119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712014-c495-4027-a8b5-6544ac2648bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.257406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712014-c495-4027-a8b5-6544ac2648bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.257688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712014-c495-4027-a8b5-6544ac2648bc admin admin] PATH_INFO: `/v3/projects/6754afdf4fcc46fbb4014f38ae3d9b73/users/4ad0be39d4374c6abb43421da6d2c722/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.272889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc712014-c495-4027-a8b5-6544ac2648bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6754afdf4fcc46fbb4014f38ae3d9b73, user_id=4ad0be39d4374c6abb43421da6d2c722, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.275602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc712014-c495-4027-a8b5-6544ac2648bc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.279135 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4517f235551f44d4852c262fc1297882'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:20.279753 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4517f235551f44d4852c262fc1297882'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:20.280352 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2918043f-19ea-450e-989b-b65b942eeb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4517f235551f44d4852c262fc1297882'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:20.286321 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:20.298961 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1359/2691] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:19 2022] DELETE /identity/v3/users/4517f235551f44d4852c262fc1297882 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:20.303401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.303694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.303978 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.322114 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:20.323333 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:20.373427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1333/2692] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:20 2022] PUT /identity/v3/projects/6754afdf4fcc46fbb4014f38ae3d9b73/users/4ad0be39d4374c6abb43421da6d2c722/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:20.379163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.379457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.379927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.385058 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1360/2693] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:20.394943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:20.396376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:20.397727 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:20.400107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:20.454319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34ea96-e048-4bb6-aaa4-032db7166fd1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.459083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1334/2694] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:20.468897 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:20.504577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.505360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.505768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.506137 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.515986 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.516718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:20.517987 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.526906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ff383-fb26-48c3-81cc-28cc29378ce5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:20.527767 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1361/2695] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:51:20.531441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eb646-850f-4787-96bc-239f8034b0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:20.532779 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eb646-850f-4787-96bc-239f8034b0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:20.533156 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eb646-850f-4787-96bc-239f8034b0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:20.533446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eb646-850f-4787-96bc-239f8034b0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:20.543002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7eb646-850f-4787-96bc-239f8034b0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:20.544436 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7eb646-850f-4787-96bc-239f8034b0d1 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:20.555224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1335/2696] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:51:21.046955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.048584 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:21.062869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc44ee-d51b-4b7d-bef1-3caff1b3ce86 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.070433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.071353 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.088900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.089312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] PATH_INFO: `/v3/projects/dfd25b0deafe47e0ad3f7c729d6c5d76` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.091140 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd25b0deafe47e0ad3f7c729d6c5d76)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:21.093292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:21.251734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:21.258465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd25b0deafe47e0ad3f7c729d6c5d76', 'request_id': 'req-76316f45-3214-4368-8039-ff13208806ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:21.266994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd25b0deafe47e0ad3f7c729d6c5d76', 'request_id': 'req-76316f45-3214-4368-8039-ff13208806ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:21.350511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd25b0deafe47e0ad3f7c729d6c5d76', 'request_id': 'req-76316f45-3214-4368-8039-ff13208806ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:21.367709 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] The token cache is being invalidate because project dfd25b0deafe47e0ad3f7c729d6c5d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:21.368375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:21.369601 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-76316f45-3214-4368-8039-ff13208806ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:21.373869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1362/2697] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:21 2022] DELETE /identity/v3/projects/dfd25b0deafe47e0ad3f7c729d6c5d76 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:21.380200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.380200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.380200 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.397980 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:21.399202 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:21.460493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1336/2698] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:21.468581 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.468898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.469193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.488445 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:21.489741 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:21.499728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1363/2699] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:21.504518 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.517604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b8b36-aa52-46b5-9a65-b57b06786e34 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.522573 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.523326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.523661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.523948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.525012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:21.525012 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:21.526627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:21.544239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1337/2700] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:51:21.546301 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.557450 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417efaf-903b-497a-bdff-773cd6d9be97 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.562453 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.563263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.563609 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.563954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.564822 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:21.566973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:21.692874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1364/2701] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:21 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:21.697077 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.707992 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9946b-ad75-45ea-86a5-b9a6a7f21353 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.780839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.781437 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.782693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.782693 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.783313 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:21.785253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:21.873088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1338/2702] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:21 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:21.874572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.885511 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a06d7-a1b5-4343-848b-46a7b42f2a67 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.891679 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.893170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.893170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.893500 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.894035 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:21.895648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:21.909820 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1365/2703] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:21.912131 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.922572 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94c2ca-026b-422b-b409-4c8273adc329 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.928752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ae1c1-a61d-45b8-862d-19ac88c3aab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.929454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae1c1-a61d-45b8-862d-19ac88c3aab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.929741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae1c1-a61d-45b8-862d-19ac88c3aab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.930021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae1c1-a61d-45b8-862d-19ac88c3aab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.933363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ae1c1-a61d-45b8-862d-19ac88c3aab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:21.933363 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ae1c1-a61d-45b8-862d-19ac88c3aab4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:21.943105 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1339/2704] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:21.947622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:21.968769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4265cc-86b2-4433-8172-2fa030348932 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:21.974596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:21.975512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:21.975802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:21.976085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] PATH_INFO: `/v3/projects/6b133be4ad8741be89ef604e2108f8f6/users/24efece02d1c468784fda0f87072da16/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:21.998585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b133be4ad8741be89ef604e2108f8f6, user_id=24efece02d1c468784fda0f87072da16, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:22.001917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:22.094198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1366/2705] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:21 2022] PUT /identity/v3/projects/6b133be4ad8741be89ef604e2108f8f6/users/24efece02d1c468784fda0f87072da16/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:22.105359 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccf40c-8c2a-480b-a7d8-6a282ed3662f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:22.105655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccf40c-8c2a-480b-a7d8-6a282ed3662f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:22.105955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccf40c-8c2a-480b-a7d8-6a282ed3662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:22.134023 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f2ccf40c-8c2a-480b-a7d8-6a282ed3662f None None] MFA Rules not processed for user `24efece02d1c468784fda0f87072da16`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:22.135254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f2ccf40c-8c2a-480b-a7d8-6a282ed3662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:22.156639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:22.206752 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1340/2706] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:22.221183 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:22.223158 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:22.322316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296fd74-8734-4b0f-bf7f-586db58e5e41 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:22.326255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:22.326976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:22.327328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:22.327698 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:22.339345 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:22.343145 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:22.353894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1367/2707] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:26.764106 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:26.764697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:26.764697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:26.795316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] MFA Rules not processed for user `4ad0be39d4374c6abb43421da6d2c722`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:26.796560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:26.867859 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1341/2708] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:26.872464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:26.873923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:26.924129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:26.940329 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d2eed-6448-4945-8455-6372bedf15ff service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:26.944883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:26.945534 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:26.945827 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:26.946111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:26.946808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:26.948700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.038159 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1368/2709] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:26 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:27.042397 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.054391 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d44cc-f1a7-4934-98a2-0fdc2b410a3b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:27.111560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:27.112221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.112505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.112817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.113500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:27.115443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.216446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1342/2710] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:27 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:27.221103 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.231956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd8dab-813d-4e14-9c75-36f345e8bec4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:27.237413 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:27.238081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.238364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.238639 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.239398 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:27.241101 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.259804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1369/2711] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:27.264144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.275336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69297009-151f-4a0e-bc8b-96aedc2adbf3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:27.280343 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:27.281376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.282282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.282534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] PATH_INFO: `/v3/projects/9a85faa949064966922e9e4ca9d341f1/users/46e22ee29b654193853528f5d323e759/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.298743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a85faa949064966922e9e4ca9d341f1, user_id=46e22ee29b654193853528f5d323e759, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:27.301451 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.401704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1343/2712] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:27 2022] PUT /identity/v3/projects/9a85faa949064966922e9e4ca9d341f1/users/46e22ee29b654193853528f5d323e759/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:27.406089 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.417297 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1ec97-1cbf-4e4d-a19d-35a24b675593 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:27.475676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:27.476357 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.476648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.476966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.477692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:27.479319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.489985 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1370/2713] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:27.494151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:27.504974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15a3b2-d331-4280-b96a-6f4643a0d23a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:27.509861 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271f723-7bdc-4427-af25-6ca6ffa43cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:27.510674 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271f723-7bdc-4427-af25-6ca6ffa43cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:27.511108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271f723-7bdc-4427-af25-6ca6ffa43cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:27.511220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271f723-7bdc-4427-af25-6ca6ffa43cd4 admin admin] PATH_INFO: `/v3/projects/9a85faa949064966922e9e4ca9d341f1/users/46e22ee29b654193853528f5d323e759/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:27.513280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271f723-7bdc-4427-af25-6ca6ffa43cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a85faa949064966922e9e4ca9d341f1, user_id=46e22ee29b654193853528f5d323e759, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:27.515938 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271f723-7bdc-4427-af25-6ca6ffa43cd4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:27.528222 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1344/2714] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:27 2022] PUT /identity/v3/projects/9a85faa949064966922e9e4ca9d341f1/users/46e22ee29b654193853528f5d323e759/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:28.987019 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.013180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7218a94-7d0a-4806-add9-ff33b68a7fde admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.149290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.150844 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.151239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.151687 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] PATH_INFO: `/v3/users/24efece02d1c468784fda0f87072da16` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.153174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24efece02d1c468784fda0f87072da16)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.155905 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.476883 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24efece02d1c468784fda0f87072da16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.488120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24efece02d1c468784fda0f87072da16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.493761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:29.590134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24efece02d1c468784fda0f87072da16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.591060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24efece02d1c468784fda0f87072da16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.592395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24efece02d1c468784fda0f87072da16'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:29.618626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1371/2715] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:28 2022] DELETE /identity/v3/users/24efece02d1c468784fda0f87072da16 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:29.620423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.621275 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.621713 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.632214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:29.648072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:29.649538 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:29.724146 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1345/2716] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:29.762714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:29.764168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:29.897425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324e28-db8d-4e5f-8f0f-7fb393130d15 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:29.906519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548ce5f-16de-4ad0-8b2a-e61182ad9a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:29.907248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ce5f-16de-4ad0-8b2a-e61182ad9a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:29.907521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ce5f-16de-4ad0-8b2a-e61182ad9a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:29.908096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ce5f-16de-4ad0-8b2a-e61182ad9a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:29.921220 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548ce5f-16de-4ad0-8b2a-e61182ad9a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:29.923597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548ce5f-16de-4ad0-8b2a-e61182ad9a38 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:29.937761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1372/2717] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:30.642049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:30.646457 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:30.660347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f89c41-b42c-4cb1-b2c9-dce50758674a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:30.685902 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:30.687135 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:30.687649 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:30.688176 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] PATH_INFO: `/v3/projects/6b133be4ad8741be89ef604e2108f8f6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:30.699141 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b133be4ad8741be89ef604e2108f8f6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:30.704059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.011026 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b133be4ad8741be89ef604e2108f8f6', 'request_id': 'req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.085169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b133be4ad8741be89ef604e2108f8f6', 'request_id': 'req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.091662 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b133be4ad8741be89ef604e2108f8f6', 'request_id': 'req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.106062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] The token cache is being invalidate because project 6b133be4ad8741be89ef604e2108f8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:31.106761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.107941 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d81f896-ef27-4e6a-8685-3b5ce3de33e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:31.109492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1346/2718] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:30 2022] DELETE /identity/v3/projects/6b133be4ad8741be89ef604e2108f8f6 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:31.115430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.115953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.116054 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.146858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:31.148170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:31.221642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1373/2719] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:31.230507 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.233117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.233117 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.249949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:31.251717 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:31.260016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1347/2720] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:31.264475 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.275166 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99029dfd-db1c-4395-901b-f287ff73d687 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.281076 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.281942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.282324 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.282694 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.283523 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:31.283930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.285723 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.297764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1374/2721] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:51:31.303064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.315428 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284d050-35a8-4415-867e-ff6695226d09 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.324109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.327913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.330382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.330978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.331990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.334256 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.447013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1348/2722] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:31 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:31.451085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.462193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3836a8-df1b-404c-b314-4eaac40c1b32 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.536230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.537046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.537336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.537617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.538337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.540274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.630111 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1375/2723] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:31 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:31.634182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.645791 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843089f-605e-4d55-9fee-e4c2bcf53a9e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.650836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.651517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.651804 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.652083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.652864 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.654800 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.669328 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1349/2724] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:31.670592 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.684765 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf127e2-8f24-4a36-98c0-4325ff1f932d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.690824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73a95-8fb4-4d36-bb9f-f71bf9d7bca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.691519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73a95-8fb4-4d36-bb9f-f71bf9d7bca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.691815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73a95-8fb4-4d36-bb9f-f71bf9d7bca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.692102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73a95-8fb4-4d36-bb9f-f71bf9d7bca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.692900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e73a95-8fb4-4d36-bb9f-f71bf9d7bca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.694714 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e73a95-8fb4-4d36-bb9f-f71bf9d7bca3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.706934 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1376/2725] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:31.711379 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:31.724728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb84ebc-7c7d-48b9-bb69-cacbf3d0e058 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:31.734072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:31.746554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.749035 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.749521 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] PATH_INFO: `/v3/projects/a8a22e18b3244c1cb1c59f5a664c658e/users/cd9d2127588d46dc96fcfbb88abbbde7/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.768336 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a22e18b3244c1cb1c59f5a664c658e, user_id=cd9d2127588d46dc96fcfbb88abbbde7, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:31.771959 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:31.855943 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1350/2726] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:31 2022] PUT /identity/v3/projects/a8a22e18b3244c1cb1c59f5a664c658e/users/cd9d2127588d46dc96fcfbb88abbbde7/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:31.868366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14c5a2-9dd1-4988-8eed-77b242c257b9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:31.868582 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14c5a2-9dd1-4988-8eed-77b242c257b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:31.868960 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14c5a2-9dd1-4988-8eed-77b242c257b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:31.910728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7c14c5a2-9dd1-4988-8eed-77b242c257b9 None None] MFA Rules not processed for user `cd9d2127588d46dc96fcfbb88abbbde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:31.912772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c14c5a2-9dd1-4988-8eed-77b242c257b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:31.919867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:32.017734 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1377/2727] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:32.032376 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:32.036295 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:32.080473 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:32.154968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6114e-da26-4987-b53a-e8e7a6fc0d88 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:32.165626 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:32.166509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:32.166798 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:32.167074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:32.177228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:32.178745 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:32.201324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1351/2728] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:34.897143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ef8ab-c890-4d3d-b3cf-d19799516bab None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:34.897143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ef8ab-c890-4d3d-b3cf-d19799516bab None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:34.897143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ef8ab-c890-4d3d-b3cf-d19799516bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:34.924554 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e0ef8ab-c890-4d3d-b3cf-d19799516bab None None] MFA Rules not processed for user `46e22ee29b654193853528f5d323e759`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:34.926245 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e0ef8ab-c890-4d3d-b3cf-d19799516bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:35.016490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1378/2729] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:35.028919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:35.031846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:35.092904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bdb3e-d8da-430d-8ea3-b9d066a2b7f2 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:35.096728 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb3626f45544ba81603a722821fce3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:35.097386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:35.097675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:35.097958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:35.099744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:35.166934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:35.168405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:35.178576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1352/2730] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:35.480055 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676005e-8405-41d7-8cca-73d85c81054f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:35.480438 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676005e-8405-41d7-8cca-73d85c81054f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:35.480762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676005e-8405-41d7-8cca-73d85c81054f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:35.508623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c676005e-8405-41d7-8cca-73d85c81054f None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:35.509900 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c676005e-8405-41d7-8cca-73d85c81054f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:35.558423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:35.575622 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1379/2731] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:51:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:35.793782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:35.795316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:35.874675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376857f2-3574-436b-bbba-6837d0a2ef02 service glance] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:35.880914 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffc930-36b7-410e-a863-272b08de7db8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:35.881524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffc930-36b7-410e-a863-272b08de7db8 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:35.881822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffc930-36b7-410e-a863-272b08de7db8 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:35.882071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffc930-36b7-410e-a863-272b08de7db8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:35.893037 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffc930-36b7-410e-a863-272b08de7db8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:35.894695 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffc930-36b7-410e-a863-272b08de7db8 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:35.903108 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1353/2732] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:51:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:51:45.899738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:46.000860 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:46.410895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:46.416397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:46.416397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:46.429585 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] MFA Rules not processed for user `4ad0be39d4374c6abb43421da6d2c722`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:46.430824 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:46.438457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1380/2733] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:50.165360 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:50.166204 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:50.167386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:50.184347 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:50.185609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:50.193985 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1354/2734] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:50.208480 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:50.222014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9034c-179b-4385-bab0-f8bafc9d22cf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:50.229094 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b248d2-409a-46b2-bd72-254ed7402bcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:50.232196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b248d2-409a-46b2-bd72-254ed7402bcc service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:50.232196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b248d2-409a-46b2-bd72-254ed7402bcc service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:50.232196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b248d2-409a-46b2-bd72-254ed7402bcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:50.245681 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b248d2-409a-46b2-bd72-254ed7402bcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:50.247632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b248d2-409a-46b2-bd72-254ed7402bcc service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:50.258568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1381/2735] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:51:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:51:51.294019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:51.295948 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:51.461826 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7e925-c10c-40d6-9c65-61c39e2bb33d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:51.478780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:51.479969 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:51.480422 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:51.480946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] PATH_INFO: `/v3/users/9f065e19b02a402fa63b9fa7cbea4f74` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:51.482440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f065e19b02a402fa63b9fa7cbea4f74)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:51.484643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:51.806785 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f065e19b02a402fa63b9fa7cbea4f74'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:51.834092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f065e19b02a402fa63b9fa7cbea4f74'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:51.849038 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f065e19b02a402fa63b9fa7cbea4f74'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:51.956173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f065e19b02a402fa63b9fa7cbea4f74'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:51.957262 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f065e19b02a402fa63b9fa7cbea4f74'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:51.967584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1355/2736] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:51 2022] DELETE /identity/v3/users/9f065e19b02a402fa63b9fa7cbea4f74 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:51.973057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:51.974095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:51.974586 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:52.008409 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:52.011955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:52.104382 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1382/2737] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:52.115892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:52.117575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:52.186157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cfebc-37d4-4561-ae63-437f67001285 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:52.201024 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:52.202199 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:52.202839 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:52.203406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:52.214692 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:52.216151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:52.227206 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1356/2738] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:53.056307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:53.057819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:53.092813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d89aa-3086-4c14-9f9f-5fb77f1e81b7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:53.098773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:53.099951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:53.099951 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:53.100277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] PATH_INFO: `/v3/projects/f6898f6872fd4bd381b8f76848ad29d8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:53.101389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6898f6872fd4bd381b8f76848ad29d8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:53.103312 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:53.294320 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6898f6872fd4bd381b8f76848ad29d8', 'request_id': 'req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.301535 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6898f6872fd4bd381b8f76848ad29d8', 'request_id': 'req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.377674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6898f6872fd4bd381b8f76848ad29d8', 'request_id': 'req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.390787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] The token cache is being invalidate because project f6898f6872fd4bd381b8f76848ad29d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:53.391083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.391742 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.393360 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1383/2739] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:53 2022] DELETE /identity/v3/projects/f6898f6872fd4bd381b8f76848ad29d8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:53.397433 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:53.408635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97316d6b-9e3a-4f37-afa3-b22f97d3eeae service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:53.424548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:53.477596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:53.478479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:53.478889 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:53.479175 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] PATH_INFO: `/v3/users/2bc7c620527d4d5d870fd28f6c0d5a7c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:53.480650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc7c620527d4d5d870fd28f6c0d5a7c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:53.482599 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:53.720505 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc7c620527d4d5d870fd28f6c0d5a7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.734412 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc7c620527d4d5d870fd28f6c0d5a7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.742097 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc7c620527d4d5d870fd28f6c0d5a7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.830650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc7c620527d4d5d870fd28f6c0d5a7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.831388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc7c620527d4d5d870fd28f6c0d5a7c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:53.838145 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1357/2740] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:53 2022] DELETE /identity/v3/users/2bc7c620527d4d5d870fd28f6c0d5a7c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:54.136073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:54.137602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:54.159070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e5d2b-0af0-4b8e-914a-74d6a1d53f86 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:54.268782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:54.272026 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.272328 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.272617 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] PATH_INFO: `/v3/users/cd9d2127588d46dc96fcfbb88abbbde7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.277098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd9d2127588d46dc96fcfbb88abbbde7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:54.278701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:54.291112 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:54.437548 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:54.439182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:54.459876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5044748-40b0-40ed-8679-d08d1542c669 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:54.471192 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:54.472099 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.474830 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.475165 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] PATH_INFO: `/v3/projects/1eb1c0fb84394a80936f23e6ee57b842` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.483619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb1c0fb84394a80936f23e6ee57b842)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:54.485613 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:54.489096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd9d2127588d46dc96fcfbb88abbbde7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.508146 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd9d2127588d46dc96fcfbb88abbbde7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.610393 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb1c0fb84394a80936f23e6ee57b842', 'request_id': 'req-074d719f-2719-42ce-8546-b8b1619fe42d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.617562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd9d2127588d46dc96fcfbb88abbbde7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.618338 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd9d2127588d46dc96fcfbb88abbbde7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.619071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5f008a70-f37a-47ec-a827-f70b049203cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd9d2127588d46dc96fcfbb88abbbde7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.645794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1384/2741] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:54 2022] DELETE /identity/v3/users/cd9d2127588d46dc96fcfbb88abbbde7 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:54.650330 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.650618 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.650895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.669995 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:54.671295 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:54.674324 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb1c0fb84394a80936f23e6ee57b842', 'request_id': 'req-074d719f-2719-42ce-8546-b8b1619fe42d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.682211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb1c0fb84394a80936f23e6ee57b842', 'request_id': 'req-074d719f-2719-42ce-8546-b8b1619fe42d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.701549 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] The token cache is being invalidate because project 1eb1c0fb84394a80936f23e6ee57b842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:54.706683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.706683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-074d719f-2719-42ce-8546-b8b1619fe42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:54.713618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1358/2742] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:54 2022] DELETE /identity/v3/projects/1eb1c0fb84394a80936f23e6ee57b842 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:54.715308 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1d4c4-c1a0-4e3f-8087-c556799e5a7a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.715608 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1d4c4-c1a0-4e3f-8087-c556799e5a7a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.715886 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1d4c4-c1a0-4e3f-8087-c556799e5a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.746584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e2e1d4c4-c1a0-4e3f-8087-c556799e5a7a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:54.747823 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e2e1d4c4-c1a0-4e3f-8087-c556799e5a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:54.757223 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1385/2743] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:54.764521 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:54.766654 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:54.771298 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1359/2744] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:54.781287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.781611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.781896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.805837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:54.805837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:54.815149 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1360/2745] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:54.820291 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:54.827339 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:54.834478 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0991f-c0f5-48cb-9ec6-7232438d6fcd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:54.840425 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:54.841911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.842331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.842622 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.843990 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:54.844315 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:54.849317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:54.869963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1361/2746] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:51:54.871048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:54.882084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304af788-abeb-4d9e-9941-cf90ea8d9e6f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:54.883811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eb47f-1ccf-4cb8-bd7f-7d8f8ce2cad3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:54.886157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:54.886958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.887342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.887686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.890534 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:54.895891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:54.895891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:54.895891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:54.895891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:54.895891 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:54.899456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:54.900983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:54.927181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1386/2747] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:51:55.021237 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1362/2748] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:54 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:55.025157 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.037516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c91d5-f00e-4ca5-b7a5-9d46d3211865 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:55.063907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:55.099118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:55.099794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.100081 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.100358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.101079 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:55.103052 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:55.304758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1387/2749] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:55 2022] POST /identity/v3/users => generated 282 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:55.313680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.324441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107214a3-3533-48cc-b54a-dbf60269a234 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:55.334382 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:55.335074 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.335367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.335656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.336406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:55.338062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:55.354953 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1363/2750] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:55.359221 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.371994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3603714-0350-4d76-9e8a-1da320eee5ce admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:55.379512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:55.381396 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.387956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.387956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.387956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:55.387956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:55.396756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1388/2751] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:55.400877 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.415743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24cfb9-3d76-4a9a-85f3-eeab349cd771 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:55.423240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efb056-3fdb-4031-bbef-13e1965eb454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:55.423240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efb056-3fdb-4031-bbef-13e1965eb454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.423240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efb056-3fdb-4031-bbef-13e1965eb454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.423240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efb056-3fdb-4031-bbef-13e1965eb454 admin admin] PATH_INFO: `/v3/projects/dfa9b35f80dd4983af5b481595611105/users/5d02d19722284231bf665e0970a9ab3b/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.441405 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efb056-3fdb-4031-bbef-13e1965eb454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa9b35f80dd4983af5b481595611105, user_id=5d02d19722284231bf665e0970a9ab3b, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:55.444498 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efb056-3fdb-4031-bbef-13e1965eb454 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:55.466127 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.470977 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:55.488653 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24e0de-f436-4b64-a946-0560a645f9f8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:55.502830 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:55.503759 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.504014 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.504283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] PATH_INFO: `/v3/projects/a8a22e18b3244c1cb1c59f5a664c658e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.506179 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a22e18b3244c1cb1c59f5a664c658e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:55.508776 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:55.550500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1364/2752] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:55 2022] PUT /identity/v3/projects/dfa9b35f80dd4983af5b481595611105/users/5d02d19722284231bf665e0970a9ab3b/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:55.561443 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.562300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.562600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.607899 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] MFA Rules not processed for user `5d02d19722284231bf665e0970a9ab3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:55.610084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:55.692589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1365/2753] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:55 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:55.697927 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.773193 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957670b-ba1c-4caf-9e5f-b292f7c41bb2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:55.780219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:55.781130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:55.781467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:55.781771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:55.782584 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:55.784762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:55.962640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a22e18b3244c1cb1c59f5a664c658e', 'request_id': 'req-f9be0b18-244c-4229-99da-ef8a989484f0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:55.980504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1366/2754] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:55 2022] POST /identity/v3/projects => generated 365 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:51:55.985231 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:55.985756 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a22e18b3244c1cb1c59f5a664c658e', 'request_id': 'req-f9be0b18-244c-4229-99da-ef8a989484f0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:55.997708 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:56.059638 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf360ed-e3b9-46e8-9f5f-8d0462f02ea8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.066189 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.067089 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.067540 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.067736 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a22e18b3244c1cb1c59f5a664c658e', 'request_id': 'req-f9be0b18-244c-4229-99da-ef8a989484f0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:56.068155 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.069048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.071775 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.083539 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] The token cache is being invalidate because project a8a22e18b3244c1cb1c59f5a664c658e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:56.083848 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:56.084674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f9be0b18-244c-4229-99da-ef8a989484f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:56.086280 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1389/2755] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:55 2022] DELETE /identity/v3/projects/a8a22e18b3244c1cb1c59f5a664c658e => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:51:56.092073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a1e30-6d8f-4314-9166-c92b2d8b2448 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.092489 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a1e30-6d8f-4314-9166-c92b2d8b2448 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.093642 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a1e30-6d8f-4314-9166-c92b2d8b2448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.115240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0e9a1e30-6d8f-4314-9166-c92b2d8b2448 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:56.116490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e9a1e30-6d8f-4314-9166-c92b2d8b2448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:56.126244 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1390/2756] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:56.134894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.135191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.135496 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.154643 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:56.156519 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:56.165346 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1391/2757] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:56.169862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.180760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712c5bb-e6da-43ca-8111-6f3a91a1c77b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.183887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1367/2758] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:55 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:51:56.188153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.192468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.192468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.192468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.192468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:51:56.192468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.192468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.197850 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.204045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:56.212447 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1392/2759] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:51:56.213686 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.218350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51080cd6-9b08-4d6d-879a-06338eeef332 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.224185 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39dead-a6d8-40f7-ad57-da2556abd5ff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.224430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.225104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.225388 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.225664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.226396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.228198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.239680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1368/2760] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:56.240268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.240976 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.241277 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.241545 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.242248 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.244008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.244191 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.255788 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e058c-3e95-479e-b3d5-27a7537343f7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.261296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.262211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.262545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.262868 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] PATH_INFO: `/v3/projects/4435c7f21e8049f793111b6edfc4272e/users/8d989c34b0d34c83a0f4e3f88602c5a0/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.285615 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4435c7f21e8049f793111b6edfc4272e, user_id=8d989c34b0d34c83a0f4e3f88602c5a0, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.288310 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.357856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1393/2761] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/projects => generated 399 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:51:56.363021 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.389955 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1369/2762] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:56 2022] PUT /identity/v3/projects/4435c7f21e8049f793111b6edfc4272e/users/8d989c34b0d34c83a0f4e3f88602c5a0/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:56.402448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.416955 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:56.450045 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af164e-a18b-48f1-9bf2-5b7ecbc94026 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.481996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af09406-2d56-45a8-a9a7-54d889f8a6df admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.488214 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.489246 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.489711 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.490174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.490931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.492553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.504958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1370/2763] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:51:56.509601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.521067 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bd8d7-3e63-47c0-a7b4-8a8071fa6a59 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.522027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.523075 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.523467 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.523826 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.524630 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.526704 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.527016 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba57cc-ebf5-4b27-829e-9db2e1ac7317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.528367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba57cc-ebf5-4b27-829e-9db2e1ac7317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.528714 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba57cc-ebf5-4b27-829e-9db2e1ac7317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.529283 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba57cc-ebf5-4b27-829e-9db2e1ac7317 admin admin] PATH_INFO: `/v3/projects/4435c7f21e8049f793111b6edfc4272e/users/8d989c34b0d34c83a0f4e3f88602c5a0/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.535400 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba57cc-ebf5-4b27-829e-9db2e1ac7317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4435c7f21e8049f793111b6edfc4272e, user_id=8d989c34b0d34c83a0f4e3f88602c5a0, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.539685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba57cc-ebf5-4b27-829e-9db2e1ac7317 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.550480 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1371/2764] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:56 2022] PUT /identity/v3/projects/4435c7f21e8049f793111b6edfc4272e/users/8d989c34b0d34c83a0f4e3f88602c5a0/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:56.563553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.563553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.563553 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.596922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] MFA Rules not processed for user `8d989c34b0d34c83a0f4e3f88602c5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:56.598455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:56.643752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1394/2765] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:51:56.645371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.648795 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:56.670178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175f9cb-d2a2-433e-a95e-60ad5be5844d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.706072 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1372/2766] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:51:56.729527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.731479 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:56.769385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.770073 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.770358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.770647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.771531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.773203 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.785355 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1395/2767] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:51:56.789306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.799911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10ffe8-ed2e-402c-a08b-94be59de1dae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.804842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.805518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.805810 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.806096 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.806833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.808565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.824456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1396/2768] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:51:56.825394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:56.828618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e019f-6b67-4b43-8b26-2bb20f8cd8c2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.833531 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193a29d-6e55-42e1-bb7a-22bbdc813494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.834580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a29d-6e55-42e1-bb7a-22bbdc813494 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.834989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a29d-6e55-42e1-bb7a-22bbdc813494 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.835358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a29d-6e55-42e1-bb7a-22bbdc813494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.836082 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db3d64-1d3b-446f-a2d5-56243ee43a2c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:56.841164 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:56.842060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.842411 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.842700 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] PATH_INFO: `/v3/projects/172a4419fe434cb387714910741171a5/users/e5bd54c13cea4015b477247cd9c01257/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:56.850501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193a29d-6e55-42e1-bb7a-22bbdc813494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.856019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193a29d-6e55-42e1-bb7a-22bbdc813494 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.862198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=172a4419fe434cb387714910741171a5, user_id=e5bd54c13cea4015b477247cd9c01257, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:56.865160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:56.866958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1373/2769] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:56.967593 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1397/2770] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:51:56 2022] PUT /identity/v3/projects/172a4419fe434cb387714910741171a5/users/e5bd54c13cea4015b477247cd9c01257/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:56.978329 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:56.979028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:56.979602 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:57.054900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] MFA Rules not processed for user `e5bd54c13cea4015b477247cd9c01257`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:57.059269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:57.163002 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1374/2771] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:57.170958 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:57.227801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:57.244201 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e53c7-aa7d-4bb0-8ed6-bcc7f19db44f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:57.248077 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fdb08-5738-4f4a-98f8-a463ba6a9360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:57.248764 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fdb08-5738-4f4a-98f8-a463ba6a9360 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:57.249104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fdb08-5738-4f4a-98f8-a463ba6a9360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:57.249412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fdb08-5738-4f4a-98f8-a463ba6a9360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:57.258894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fdb08-5738-4f4a-98f8-a463ba6a9360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:57.261607 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fdb08-5738-4f4a-98f8-a463ba6a9360 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:57.278153 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1398/2772] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:58.644751 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:58.731282 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc13fd-b82a-41e8-9a9b-7f712f3f2790 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:58.737481 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:58.738508 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:58.738644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:58.738911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] PATH_INFO: `/v3/users/5d02d19722284231bf665e0970a9ab3b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:58.740144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d02d19722284231bf665e0970a9ab3b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:58.742084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:58.751689 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:51:58.939762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d02d19722284231bf665e0970a9ab3b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:58.953378 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d02d19722284231bf665e0970a9ab3b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:58.961783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d02d19722284231bf665e0970a9ab3b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.028946 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d02d19722284231bf665e0970a9ab3b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.029594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d02d19722284231bf665e0970a9ab3b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.038904 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1375/2773] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:58 2022] DELETE /identity/v3/users/5d02d19722284231bf665e0970a9ab3b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:59.042596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:59.043143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:59.043143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:59.062954 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:51:59.064410 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:59.142348 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1399/2774] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:51:59.149963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:59.157497 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:59.220744 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137828a-77c9-482d-8786-e430db7af3f2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:59.224345 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:59.225018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:59.225305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:59.225580 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:59.235157 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:59.236578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:59.246931 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1376/2775] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:51:59.675898 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:59.677309 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:51:59.691133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37541af1-b6c1-4eb9-b921-644d007e3e1f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:59.695853 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:59.696771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:59.697060 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:59.697340 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] PATH_INFO: `/v3/projects/dfa9b35f80dd4983af5b481595611105` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:59.698543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa9b35f80dd4983af5b481595611105)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:59.700515 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:51:59.825983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa9b35f80dd4983af5b481595611105', 'request_id': 'req-0195ee6c-7a2f-4b06-a10d-7de1109c048a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.828672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa9b35f80dd4983af5b481595611105', 'request_id': 'req-0195ee6c-7a2f-4b06-a10d-7de1109c048a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.901384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa9b35f80dd4983af5b481595611105', 'request_id': 'req-0195ee6c-7a2f-4b06-a10d-7de1109c048a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.912772 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] The token cache is being invalidate because project dfa9b35f80dd4983af5b481595611105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:51:59.913068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.913696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:51:59.915229 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1400/2776] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:51:59 2022] DELETE /identity/v3/projects/dfa9b35f80dd4983af5b481595611105 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:51:59.919209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:51:59.929441 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77a13-72ab-4e5e-9e55-7763cebefa15 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:51:59.987287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:51:59.988174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:51:59.988465 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:51:59.988780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] PATH_INFO: `/v3/users/8d989c34b0d34c83a0f4e3f88602c5a0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:51:59.990358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d989c34b0d34c83a0f4e3f88602c5a0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:51:59.992064 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:00.091710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:00.184007 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d989c34b0d34c83a0f4e3f88602c5a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:00.196299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d989c34b0d34c83a0f4e3f88602c5a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:00.204307 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d989c34b0d34c83a0f4e3f88602c5a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:00.292174 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d989c34b0d34c83a0f4e3f88602c5a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:00.292386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-da60306a-b95a-45c1-915c-6cfab7bd191f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d989c34b0d34c83a0f4e3f88602c5a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:00.306822 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1377/2777] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:51:59 2022] DELETE /identity/v3/users/8d989c34b0d34c83a0f4e3f88602c5a0 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:00.778648 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:00.780130 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:00.795234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ee6c-7a2f-4b06-a10d-7de1109c048a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:00.855891 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:00.856783 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:00.857068 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:00.857345 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] PATH_INFO: `/v3/projects/4435c7f21e8049f793111b6edfc4272e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:00.859215 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4435c7f21e8049f793111b6edfc4272e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:00.861143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:00.867260 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:00.970253 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4435c7f21e8049f793111b6edfc4272e', 'request_id': 'req-651ad013-d500-49ef-96a0-37ccf8d699f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:00.976380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4435c7f21e8049f793111b6edfc4272e', 'request_id': 'req-651ad013-d500-49ef-96a0-37ccf8d699f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:01.054939 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4435c7f21e8049f793111b6edfc4272e', 'request_id': 'req-651ad013-d500-49ef-96a0-37ccf8d699f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:01.065910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] The token cache is being invalidate because project 4435c7f21e8049f793111b6edfc4272e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:01.066207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:01.066917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-651ad013-d500-49ef-96a0-37ccf8d699f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:01.068501 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1401/2778] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:00 2022] DELETE /identity/v3/projects/4435c7f21e8049f793111b6edfc4272e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:01.073554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.074062 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.074175 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.092539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:01.094576 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:01.153917 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1378/2779] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:52:01.163549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.163852 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.164165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.182160 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:01.183492 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:01.191626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1402/2780] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:01.195807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.206780 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20edaf6-c5e5-4120-ad92-8f2a11db4ff7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.211453 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.212109 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.212399 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.212701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.213589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:52:01.213901 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.215544 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:01.227900 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1379/2781] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:52:01.238576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.247769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78d318-45a3-4875-af84-286eb64d5411 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.252569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.253258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.253549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.253838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.254536 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.256518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:01.397031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1403/2782] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:52:01 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:01.401229 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.411697 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5735378-d547-45ba-9cc0-dd7a9fe42ebd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.471048 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.471730 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.472020 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.472302 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.473087 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.475018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:01.570267 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1380/2783] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:52:01 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:01.574407 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.587206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f2f-b5ce-4df7-b2f3-ab20909564a8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.591927 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.592929 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.593336 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.593769 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.594564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.596293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:01.606705 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1404/2784] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:01.610776 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.621184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a17aa-6ffc-4295-8cc3-479c82573ea6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.626575 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d94c7-8690-46dd-866e-853a306b4120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.627243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d94c7-8690-46dd-866e-853a306b4120 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.627529 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d94c7-8690-46dd-866e-853a306b4120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.627808 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d94c7-8690-46dd-866e-853a306b4120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.628545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d94c7-8690-46dd-866e-853a306b4120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.630177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d94c7-8690-46dd-866e-853a306b4120 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:01.644358 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1381/2785] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:01.645786 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.656478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43176b6f-0cb8-4145-8ccb-359cf9113416 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.661316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.662205 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.662494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.662774 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] PATH_INFO: `/v3/projects/d86b0c7c51f545d6bf02b371b57cea41/users/7f9355443c9545b6b09cd9294218c08f/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.679662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86b0c7c51f545d6bf02b371b57cea41, user_id=7f9355443c9545b6b09cd9294218c08f, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.682375 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:01.760156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1405/2786] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:01 2022] PUT /identity/v3/projects/d86b0c7c51f545d6bf02b371b57cea41/users/7f9355443c9545b6b09cd9294218c08f/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:01.771134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.771429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.771738 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.791984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:01.799272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] MFA Rules not processed for user `7f9355443c9545b6b09cd9294218c08f`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:01.800490 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:01.868815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1382/2787] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:01.869808 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:01.939384 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b411-6b47-46c4-aff5-24fb1e200617 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:01.944097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:01.944902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:01.945196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:01.945497 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:01.946195 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:01.948108 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.110680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1406/2788] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:52:01 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:52:02.114787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.126551 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff28d79-9448-4b0f-b76d-0b5acf9b57a1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.181996 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.182668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.182960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.183243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.183956 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.185906 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.297825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1383/2789] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:52:02 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:02.301565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.313446 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9498-f6b8-4055-8ea6-a747ad9f0218 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.318139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.318812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.319104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.319389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.320120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.321758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.333208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1407/2790] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:02.337440 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.347550 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9a8e0-bdfe-484c-a14d-96881adfdf26 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.352112 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.353032 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.353322 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.353604 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] PATH_INFO: `/v3/projects/d45109d839654412aa0a34e730eb3790/users/dbd6e9bbc7ab4183bc8ceed8fba3e773/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.368574 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45109d839654412aa0a34e730eb3790, user_id=dbd6e9bbc7ab4183bc8ceed8fba3e773, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.371306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.474778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1384/2791] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:02 2022] PUT /identity/v3/projects/d45109d839654412aa0a34e730eb3790/users/dbd6e9bbc7ab4183bc8ceed8fba3e773/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:02.478991 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.485930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:02.489919 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c80bf-1eb0-4471-9294-ca9e51bee4e8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.548206 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a7274-b8d9-43a9-b552-407121493590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.548906 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a7274-b8d9-43a9-b552-407121493590 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.549196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a7274-b8d9-43a9-b552-407121493590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.549477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a7274-b8d9-43a9-b552-407121493590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.550212 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a7274-b8d9-43a9-b552-407121493590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.551811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a7274-b8d9-43a9-b552-407121493590 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.562746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1408/2792] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:02.566743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.577639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52df77-5d68-46be-a1cb-b21f68c80c08 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.582648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.583543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.583828 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.584107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] PATH_INFO: `/v3/projects/d45109d839654412aa0a34e730eb3790/users/dbd6e9bbc7ab4183bc8ceed8fba3e773/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.586195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45109d839654412aa0a34e730eb3790, user_id=dbd6e9bbc7ab4183bc8ceed8fba3e773, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.588871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.601459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1385/2793] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:02 2022] PUT /identity/v3/projects/d45109d839654412aa0a34e730eb3790/users/dbd6e9bbc7ab4183bc8ceed8fba3e773/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:02.613652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.613652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.613652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.788804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] MFA Rules not processed for user `dbd6e9bbc7ab4183bc8ceed8fba3e773`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:02.788804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:02.788804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1409/2794] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:02.789084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.789084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:02.789084 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774111e-59f7-4837-bdd2-f19603482ec2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.791834 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a09e85-8463-4f21-93af-6dc49f7ae690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.792488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a09e85-8463-4f21-93af-6dc49f7ae690 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.792819 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a09e85-8463-4f21-93af-6dc49f7ae690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.793101 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a09e85-8463-4f21-93af-6dc49f7ae690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.800879 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:02.803068 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a09e85-8463-4f21-93af-6dc49f7ae690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.804496 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a09e85-8463-4f21-93af-6dc49f7ae690 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:02.814979 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1386/2795] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:02.844173 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:02.906228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c16be-02eb-4e81-a217-f8256eeb7489 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:02.911083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:02.911953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:02.912239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:02.912520 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] PATH_INFO: `/v3/users/7f9355443c9545b6b09cd9294218c08f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:02.913758 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9355443c9545b6b09cd9294218c08f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:02.915688 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:03.065029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:03.196667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9355443c9545b6b09cd9294218c08f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:03.204370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9355443c9545b6b09cd9294218c08f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:03.308344 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9355443c9545b6b09cd9294218c08f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:03.309022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9355443c9545b6b09cd9294218c08f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:03.309627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9355443c9545b6b09cd9294218c08f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:03.328740 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1410/2796] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:52:02 2022] DELETE /identity/v3/users/7f9355443c9545b6b09cd9294218c08f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:03.333248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:03.333736 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:03.333855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:03.355632 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:03.356887 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:03.419247 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1387/2797] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:03.429524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:03.430973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:03.490403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18697765-0f5d-481f-8d6b-11ef4e61808c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:03.494251 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:03.494910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:03.495200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:03.495481 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:03.505406 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:03.506855 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:03.517313 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1411/2798] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:04.099123 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:04.100658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:04.113488 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f08118-c10d-4551-b9cf-12a01640e646 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:04.118467 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:04.119367 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:04.119655 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:04.119940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] PATH_INFO: `/v3/projects/d86b0c7c51f545d6bf02b371b57cea41` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:04.122253 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86b0c7c51f545d6bf02b371b57cea41)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:04.128276 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:04.257926 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86b0c7c51f545d6bf02b371b57cea41', 'request_id': 'req-aa842eb7-cc65-4003-8b7f-967f759d5d1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.352387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86b0c7c51f545d6bf02b371b57cea41', 'request_id': 'req-aa842eb7-cc65-4003-8b7f-967f759d5d1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.358595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86b0c7c51f545d6bf02b371b57cea41', 'request_id': 'req-aa842eb7-cc65-4003-8b7f-967f759d5d1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.369989 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] The token cache is being invalidate because project d86b0c7c51f545d6bf02b371b57cea41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:04.370281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.370912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.372446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1388/2799] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:04 2022] DELETE /identity/v3/projects/d86b0c7c51f545d6bf02b371b57cea41 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:04.377064 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:04.387226 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d747fa1-ee9a-4aa8-a632-cabd58ac5e60 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:04.448356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:04.449268 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:04.449556 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:04.449835 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] PATH_INFO: `/v3/users/dbd6e9bbc7ab4183bc8ceed8fba3e773` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:04.451042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd6e9bbc7ab4183bc8ceed8fba3e773)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:04.453888 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:04.711417 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd6e9bbc7ab4183bc8ceed8fba3e773'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.713602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:04.722537 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd6e9bbc7ab4183bc8ceed8fba3e773'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.822664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd6e9bbc7ab4183bc8ceed8fba3e773'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.823307 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd6e9bbc7ab4183bc8ceed8fba3e773'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.823918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9e6c0f7-de68-44a5-995b-f1d06f320ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd6e9bbc7ab4183bc8ceed8fba3e773'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:04.843561 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1412/2800] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:52:04 2022] DELETE /identity/v3/users/dbd6e9bbc7ab4183bc8ceed8fba3e773 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:05.333270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:05.334704 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:05.348515 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842eb7-cc65-4003-8b7f-967f759d5d1d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:05.408585 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:05.409643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:05.410014 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:05.410300 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] PATH_INFO: `/v3/projects/d45109d839654412aa0a34e730eb3790` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:05.411532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45109d839654412aa0a34e730eb3790)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:05.413504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:05.439619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:05.612698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45109d839654412aa0a34e730eb3790', 'request_id': 'req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:05.709459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45109d839654412aa0a34e730eb3790', 'request_id': 'req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:05.715395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45109d839654412aa0a34e730eb3790', 'request_id': 'req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:05.726588 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] The token cache is being invalidate because project d45109d839654412aa0a34e730eb3790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:05.726876 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:05.727578 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b2fc5247-c3d3-4fac-a6d8-2f465d55f714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:05.729142 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1389/2801] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:05 2022] DELETE /identity/v3/projects/d45109d839654412aa0a34e730eb3790 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:05.734596 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:05.735092 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:05.735211 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:05.756394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:05.756394 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:05.823984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1413/2802] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:05.832169 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:05.836837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:05.836837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:05.852081 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:05.853243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:05.862167 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1390/2803] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:52:05.866430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:05.876928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785e1a4-823e-4272-a721-4cf98c48c4d4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:05.881571 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:05.882249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:05.882512 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:05.882789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:05.883516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:52:05.883825 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:05.885490 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:05.898641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1414/2804] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:52:05.904846 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:05.915220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60a5ed-3a78-48e1-9f4e-a597000215ec None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:05.920085 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:05.921972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:05.921972 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:05.922385 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:05.923013 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:05.924982 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.030805 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1391/2805] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:52:05 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:06.035199 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.045532 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043b1e0-1515-49cc-8c76-fa90a848665e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.113029 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.114086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.114379 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.114667 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.115371 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.117387 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.202710 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1415/2806] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:52:06 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:06.204245 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.215006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72aba8-c852-4ed1-afb3-0b758529a865 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.221285 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.222042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.222333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.222611 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.223348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.225033 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.240429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1392/2807] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:06.242632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.253304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b63160-9c53-463c-acc2-bb5bdef6b7ae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.258193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4b277-20eb-4119-a3b0-59eed18a9749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.258854 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4b277-20eb-4119-a3b0-59eed18a9749 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.259139 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4b277-20eb-4119-a3b0-59eed18a9749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.259418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4b277-20eb-4119-a3b0-59eed18a9749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.260170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4b277-20eb-4119-a3b0-59eed18a9749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.261831 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4b277-20eb-4119-a3b0-59eed18a9749 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.272869 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1416/2808] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:06.277152 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.288100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569aa2-fa87-44df-bfa9-b116e7408f43 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.292782 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.293709 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.293999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.294281 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] PATH_INFO: `/v3/projects/4056d1942b3640a397ea279b4e18bb13/users/112007efeffb43979e331ddf1895adc0/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.308957 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4056d1942b3640a397ea279b4e18bb13, user_id=112007efeffb43979e331ddf1895adc0, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.311650 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.417334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1393/2809] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:06 2022] PUT /identity/v3/projects/4056d1942b3640a397ea279b4e18bb13/users/112007efeffb43979e331ddf1895adc0/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:52:06.427931 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.428230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.428518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.456084 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] MFA Rules not processed for user `112007efeffb43979e331ddf1895adc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:06.457399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:06.480647 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:06.523818 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1417/2810] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:06.528305 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.591184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def15e7-19c9-41c9-ba35-021d0bd8add2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.596934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.597596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.597883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.598195 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.598882 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.600818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.693663 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1394/2811] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:52:06 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:06.697870 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.709632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff45b7-cfaa-469f-8e33-96b72990813f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.765573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.766228 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.766444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.766720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.767416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.769341 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.881131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1418/2812] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:52:06 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:06.884998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.895487 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28f2ec-e283-4aa8-8609-109e70d5e114 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.900922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.900922 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.901184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.901396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.902128 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.903718 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.914815 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1395/2813] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:06.918907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.929829 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce06c8-1248-461c-bf9d-e7618e387ece admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.934466 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451e5c7-05a8-4019-91f5-f35d33f46823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.935124 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e5c7-05a8-4019-91f5-f35d33f46823 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.935414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e5c7-05a8-4019-91f5-f35d33f46823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.935697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e5c7-05a8-4019-91f5-f35d33f46823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.936627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451e5c7-05a8-4019-91f5-f35d33f46823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.938256 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451e5c7-05a8-4019-91f5-f35d33f46823 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.948156 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1419/2814] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:06.952130 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:06.962342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc891-a830-4db4-a587-6137f1e3b308 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:06.966937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:06.968387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:06.968673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:06.968978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] PATH_INFO: `/v3/projects/20948e9f518545b08758cd9fee7363db/users/54b7aca5698e4ed59f3dd9b97781caef/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:06.983855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20948e9f518545b08758cd9fee7363db, user_id=54b7aca5698e4ed59f3dd9b97781caef, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:06.986561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:06.989644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:07.073816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1396/2815] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:06 2022] PUT /identity/v3/projects/20948e9f518545b08758cd9fee7363db/users/54b7aca5698e4ed59f3dd9b97781caef/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:07.086208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.086507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.086787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.115034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] MFA Rules not processed for user `54b7aca5698e4ed59f3dd9b97781caef`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:07.116249 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.126639 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.128374 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.183672 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1420/2816] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:52:07.188414 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.194569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f08135-9382-4ceb-8393-c5decab77335 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.198801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2f674-7339-4bd3-964a-8c949d6f11e2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.199243 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.200122 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.200409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.200712 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] PATH_INFO: `/v3/users/e5bd54c13cea4015b477247cd9c01257` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.202586 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5bd54c13cea4015b477247cd9c01257)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.204565 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.204825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.205242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.205533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.205813 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.206510 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.208403 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.304668 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1421/2817] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:52:07 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:52:07.308395 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.320468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:07.378739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777b5a2-b62c-48f9-8e8e-0a579d39f328 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.383918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.384597 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.384918 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.385198 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.385941 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.388098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.443579 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5bd54c13cea4015b477247cd9c01257'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:07.456634 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5bd54c13cea4015b477247cd9c01257'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:07.459178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1422/2818] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:52:07 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:52:07.463033 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.463703 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5bd54c13cea4015b477247cd9c01257'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:07.472967 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c90de-eece-47c8-9c39-3167818c5092 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.477768 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.478425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.478717 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.478996 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.479729 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.481367 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.491408 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1423/2819] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:52:07.495288 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.505637 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219adc7b-007c-466e-b20f-a2a60f6f674e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.510240 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.511120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.515071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.515071 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] PATH_INFO: `/v3/projects/18fe8efa129245f488f5a7b5229a6a8a/users/10d8b05c81374fc79afb3db34199b518/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.526780 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fe8efa129245f488f5a7b5229a6a8a, user_id=10d8b05c81374fc79afb3db34199b518, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.529444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.571500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5bd54c13cea4015b477247cd9c01257'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:07.572139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe12aa72-f193-4fd5-a92c-eee913ff14c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5bd54c13cea4015b477247cd9c01257'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:07.580059 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1397/2820] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:52:07 2022] DELETE /identity/v3/users/e5bd54c13cea4015b477247cd9c01257 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:52:07.584434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.584760 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.585044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.601589 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:07.602817 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.611801 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1424/2821] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:07 2022] PUT /identity/v3/projects/18fe8efa129245f488f5a7b5229a6a8a/users/10d8b05c81374fc79afb3db34199b518/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:52:07.617549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.619129 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.670045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1398/2822] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:52:07.680240 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.684567 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.690358 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f590e-c22f-4361-81c1-80a3dba74e23 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.695733 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.700880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.700880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.700880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.700880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.700880 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.706448 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:07.715999 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1425/2823] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:52:07.716868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.730099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90bb2-2943-47a0-a68b-3de7a8a3e205 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.736345 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e13ca-b22b-49af-b9ef-47eefffbd3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.741086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e13ca-b22b-49af-b9ef-47eefffbd3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.741086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e13ca-b22b-49af-b9ef-47eefffbd3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.741086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e13ca-b22b-49af-b9ef-47eefffbd3cd admin admin] PATH_INFO: `/v3/projects/18fe8efa129245f488f5a7b5229a6a8a/users/10d8b05c81374fc79afb3db34199b518/roles/e4cb7641166847b497698b97084ca96b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.741086 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e13ca-b22b-49af-b9ef-47eefffbd3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fe8efa129245f488f5a7b5229a6a8a, user_id=10d8b05c81374fc79afb3db34199b518, role_id=e4cb7641166847b497698b97084ca96b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.742590 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e13ca-b22b-49af-b9ef-47eefffbd3cd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.751947 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66312f-bbe2-49d6-b5aa-4589068c9e7a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.755319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1426/2824] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:07 2022] PUT /identity/v3/projects/18fe8efa129245f488f5a7b5229a6a8a/users/10d8b05c81374fc79afb3db34199b518/roles/e4cb7641166847b497698b97084ca96b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:07.765422 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.765718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.766306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.792945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] MFA Rules not processed for user `10d8b05c81374fc79afb3db34199b518`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:07.794207 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.808951 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.809600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.809885 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.810180 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.865811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1427/2825] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:52:07.871787 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.873317 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.877575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:07.879042 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:07.884875 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1399/2826] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:52:07.938468 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c4f18-a9ad-4542-8365-c6bf185c5caf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:07.942263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:07.943022 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:07.943317 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:07.943599 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:07.953889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:07.955337 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:07.961722 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:07.971956 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1428/2827] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:52:08.083984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:08.085506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:08.098627 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8d570-727b-4a09-b9a2-f35c4c3d2e16 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:08.102316 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:08.102967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:08.103254 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:08.103642 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:08.105492 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:08.168532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:08.170049 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:08.179287 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1400/2828] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:08.362441 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:08.364061 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:08.390718 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17122a82-c926-4c5f-bed9-e754144cbaa8 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:08.395901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:08.396809 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:08.397099 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:08.397430 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] PATH_INFO: `/v3/projects/172a4419fe434cb387714910741171a5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:08.398628 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172a4419fe434cb387714910741171a5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:08.400524 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:08.504856 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172a4419fe434cb387714910741171a5', 'request_id': 'req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:08.511264 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172a4419fe434cb387714910741171a5', 'request_id': 'req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:08.611462 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172a4419fe434cb387714910741171a5', 'request_id': 'req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:08.627327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] The token cache is being invalidate because project 172a4419fe434cb387714910741171a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:08.628174 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:08.628615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fb65da7-9c83-47f3-bce7-ee86cdc959a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:08.630680 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1429/2829] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:08 2022] DELETE /identity/v3/projects/172a4419fe434cb387714910741171a5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:12.540501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:12.551541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180524-e776-4a18-bbe8-c1c725a75588 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:12.567986 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:12.608187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:12.609118 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:12.609406 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:12.609872 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] PATH_INFO: `/v3/users/112007efeffb43979e331ddf1895adc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:12.610880 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=112007efeffb43979e331ddf1895adc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:12.612855 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:12.928086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112007efeffb43979e331ddf1895adc0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:12.939661 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112007efeffb43979e331ddf1895adc0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:12.946338 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112007efeffb43979e331ddf1895adc0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.034177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112007efeffb43979e331ddf1895adc0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.034807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112007efeffb43979e331ddf1895adc0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.041348 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1401/2830] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:52:12 2022] DELETE /identity/v3/users/112007efeffb43979e331ddf1895adc0 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:13.046150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:13.046684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:13.046784 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:13.064189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:13.065424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:13.114910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:13.124148 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1430/2831] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:13.134160 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:13.135595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:13.193636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969bdf-78d6-4a0c-a63b-5528489c52b6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:13.197241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:13.197883 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:13.198177 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:13.198459 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:13.208346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:13.209836 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:13.220090 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1402/2832] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:13.702027 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:13.703842 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:13.716279 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aabb9c-90f2-4a30-8cab-3bec0f0b298a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:13.721862 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:13.722731 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:13.723149 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:13.723478 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] PATH_INFO: `/v3/projects/4056d1942b3640a397ea279b4e18bb13` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:13.724693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4056d1942b3640a397ea279b4e18bb13)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:13.726662 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:13.833533 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4056d1942b3640a397ea279b4e18bb13', 'request_id': 'req-2ab013a2-49aa-4362-a0cc-7355dd669e90'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.839674 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4056d1942b3640a397ea279b4e18bb13', 'request_id': 'req-2ab013a2-49aa-4362-a0cc-7355dd669e90'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.947945 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4056d1942b3640a397ea279b4e18bb13', 'request_id': 'req-2ab013a2-49aa-4362-a0cc-7355dd669e90'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.959916 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] The token cache is being invalidate because project 4056d1942b3640a397ea279b4e18bb13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:13.960232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.960895 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:13.962474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1431/2833] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:13 2022] DELETE /identity/v3/projects/4056d1942b3640a397ea279b4e18bb13 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:13.966605 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:13.977098 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edfc29-d9e8-447e-971b-3bae15c99bb4 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:14.017944 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:14.035008 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:14.036047 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:14.036333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:14.036614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] PATH_INFO: `/v3/users/54b7aca5698e4ed59f3dd9b97781caef` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:14.037890 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b7aca5698e4ed59f3dd9b97781caef)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:14.039789 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:14.222664 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b7aca5698e4ed59f3dd9b97781caef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.234499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b7aca5698e4ed59f3dd9b97781caef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.241293 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b7aca5698e4ed59f3dd9b97781caef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.308116 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b7aca5698e4ed59f3dd9b97781caef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.308806 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b7aca5698e4ed59f3dd9b97781caef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.315342 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1403/2834] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:52:13 2022] DELETE /identity/v3/users/54b7aca5698e4ed59f3dd9b97781caef => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:14.708454 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:14.710170 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:14.725505 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab013a2-49aa-4362-a0cc-7355dd669e90 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:14.784911 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:14.785775 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:14.786062 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:14.786339 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] PATH_INFO: `/v3/projects/20948e9f518545b08758cd9fee7363db` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:14.787549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20948e9f518545b08758cd9fee7363db)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:14.789474 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:14.895964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20948e9f518545b08758cd9fee7363db', 'request_id': 'req-b34b676a-8a08-40a2-ba86-c2b265a89f74'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.902624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20948e9f518545b08758cd9fee7363db', 'request_id': 'req-b34b676a-8a08-40a2-ba86-c2b265a89f74'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.969218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20948e9f518545b08758cd9fee7363db', 'request_id': 'req-b34b676a-8a08-40a2-ba86-c2b265a89f74'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.979730 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] The token cache is being invalidate because project 20948e9f518545b08758cd9fee7363db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:14.980049 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.980669 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:14.982239 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1432/2835] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:14 2022] DELETE /identity/v3/projects/20948e9f518545b08758cd9fee7363db => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:14.987126 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:14.988635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:15.002583 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576c999-d576-40cf-adb7-fd9785a9b18c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:15.062685 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:15.064439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:15.064439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:15.064439 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] PATH_INFO: `/v3/users/10d8b05c81374fc79afb3db34199b518` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:15.065635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d8b05c81374fc79afb3db34199b518)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:15.068018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:15.291964 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d8b05c81374fc79afb3db34199b518'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:15.303408 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:15.304630 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d8b05c81374fc79afb3db34199b518'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:15.312224 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d8b05c81374fc79afb3db34199b518'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:15.382040 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d8b05c81374fc79afb3db34199b518'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:15.382658 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-eacc5ace-f55f-43eb-86d7-c5e57a408501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d8b05c81374fc79afb3db34199b518'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:15.389073 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1404/2836] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:52:14 2022] DELETE /identity/v3/users/10d8b05c81374fc79afb3db34199b518 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:15.865832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:15.866302 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:15.880315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b676a-8a08-40a2-ba86-c2b265a89f74 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:15.889294 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:15.940435 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:15.941334 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:15.941623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:15.941910 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] PATH_INFO: `/v3/projects/18fe8efa129245f488f5a7b5229a6a8a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:15.943095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] RBAC: Authorizing `identity:delete_project(project_id=18fe8efa129245f488f5a7b5229a6a8a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:15.945104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.089861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18fe8efa129245f488f5a7b5229a6a8a', 'request_id': 'req-aadf39fb-c70e-40b7-b0e6-7edb722b222b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:16.096418 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18fe8efa129245f488f5a7b5229a6a8a', 'request_id': 'req-aadf39fb-c70e-40b7-b0e6-7edb722b222b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:16.158138 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18fe8efa129245f488f5a7b5229a6a8a', 'request_id': 'req-aadf39fb-c70e-40b7-b0e6-7edb722b222b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:16.169397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] The token cache is being invalidate because project 18fe8efa129245f488f5a7b5229a6a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:52:16.169693 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:16.170570 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-aadf39fb-c70e-40b7-b0e6-7edb722b222b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:52:16.172097 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1433/2837] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:52:15 2022] DELETE /identity/v3/projects/18fe8efa129245f488f5a7b5229a6a8a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:16.178104 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.178565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.178680 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.197998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:16.199280 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:16.258778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1405/2838] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:16.267997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.268762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.268762 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.287425 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:16.288649 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:16.297281 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1434/2839] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:16.303094 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.313264 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eff50-9cc0-4c7f-8e0d-c5c5b3cc72a2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.317873 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.318527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.318812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.319092 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.319824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:52:16.320136 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.321771 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.334277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1406/2840] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:52:16.340147 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.350290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d0e1c-bcb0-47bf-85d8-d44d79249e0f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.354966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.355641 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.355930 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.356298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.357050 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.359620 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.446938 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1435/2841] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:52:16 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:16.450999 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.462163 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de8caf-9d63-4df5-bfb6-190d5ebe554c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.517304 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.518019 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.518248 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.518524 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.519212 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.521114 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.598499 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1407/2842] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:52:16 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:52:16.602657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.613247 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be2eba-f475-42ba-acb7-67d1473f458f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.617971 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.618627 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.618949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.619234 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.619962 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.621611 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.632608 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1436/2843] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:52:16.636493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.647384 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef1412-db38-4cb0-9208-fcaca6805030 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.652233 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6fa59-f96a-4f1b-bb1f-3a3471f679ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.652636 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6fa59-f96a-4f1b-bb1f-3a3471f679ee admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.652960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6fa59-f96a-4f1b-bb1f-3a3471f679ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.653272 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6fa59-f96a-4f1b-bb1f-3a3471f679ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.654004 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6fa59-f96a-4f1b-bb1f-3a3471f679ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.655672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6fa59-f96a-4f1b-bb1f-3a3471f679ee admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.665500 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1408/2844] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:52:16.669479 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.679541 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bd667-b7ca-4406-ad0f-a1657de589a9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.684242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.685141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.685423 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.686200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] PATH_INFO: `/v3/projects/a8c3be0d7a0c46d790a0eefd632a6a83/users/3e52648bc557407db1f2e44727fbde32/roles/19944ab23542462988b68290167ac622` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.700875 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c3be0d7a0c46d790a0eefd632a6a83, user_id=3e52648bc557407db1f2e44727fbde32, role_id=19944ab23542462988b68290167ac622)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.704434 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.805907 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1437/2845] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:52:16 2022] PUT /identity/v3/projects/a8c3be0d7a0c46d790a0eefd632a6a83/users/3e52648bc557407db1f2e44727fbde32/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:52:16.811653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dc3ff-f972-45fc-9b64-7f896f5de54e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.812021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dc3ff-f972-45fc-9b64-7f896f5de54e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.812314 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dc3ff-f972-45fc-9b64-7f896f5de54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.829936 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-500dc3ff-f972-45fc-9b64-7f896f5de54e None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:16.831124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-500dc3ff-f972-45fc-9b64-7f896f5de54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:16.872017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:16.889446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1409/2846] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:16.899263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:16.901098 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:16.947811 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:16.967963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97edb-1563-44c5-aa61-898c6444b80f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:16.967963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:16.967963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:16.967963 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:16.968401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:16.977315 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:16.978746 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:16.989632 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1438/2847] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:18.425221 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:18.425847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:18.425847 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:18.454446 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] MFA Rules not processed for user `3e52648bc557407db1f2e44727fbde32`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:18.455715 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:18.521228 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1410/2848] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:18.536899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:18.536899 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:18.591436 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9734d8-58f0-4bfb-b01e-5151bce3d89b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:18.595494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1b1f-b931-4e4d-be05-40d71886e991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:18.596115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1b1f-b931-4e4d-be05-40d71886e991 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:18.596402 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1b1f-b931-4e4d-be05-40d71886e991 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:18.596701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1b1f-b931-4e4d-be05-40d71886e991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:18.605087 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db1b1f-b931-4e4d-be05-40d71886e991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:18.606516 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db1b1f-b931-4e4d-be05-40d71886e991 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:18.616133 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1439/2849] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:19.697537 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:52:19.710424 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6350-376f-4e38-ade0-bb170a8c6325 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:52:19.715734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:52:19.716818 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:19.717423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:19.717884 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:19.721892 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:19.790596 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:52:19.791958 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e78d8-a1cd-4088-85aa-6eb8e16fa97d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:52:19.802862 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1411/2850] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:52:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:52:20.607416 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3b261-18eb-4782-9e3b-b615dd075ddf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:20.607966 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3b261-18eb-4782-9e3b-b615dd075ddf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:20.608354 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3b261-18eb-4782-9e3b-b615dd075ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:20.626095 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-53e3b261-18eb-4782-9e3b-b615dd075ddf None None] MFA Rules not processed for user `3e52648bc557407db1f2e44727fbde32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:20.627404 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53e3b261-18eb-4782-9e3b-b615dd075ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:20.634575 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1440/2851] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:29.798181 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=68 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:30.620602 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=38 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:52:38.849683 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:52:38.850211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:52:38.850430 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:52:38.867562 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] MFA Rules not processed for user `3e52648bc557407db1f2e44727fbde32`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:52:38.868801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:52:38.876219 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1412/2852] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:52:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:52:48.862814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:01.782626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:01.783143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:01.783390 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:01.801953 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:01.803213 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:01.859283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1441/2853] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Jul 18 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:01.899870 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:01.901309 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:01.971261 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29a69a-d16f-4093-a1d8-603ce3f5c9ea None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:01.976594 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:01.977270 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:01.977561 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:01.977845 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:01.986974 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:01.988559 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:01.997691 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1413/2854] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:53:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:53:09.557815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:09.559878 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:09.574267 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3c141-1f55-4a52-9c02-56ea6a0b0377 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:09.579105 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:09.579858 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:09.580150 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:09.580564 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:09.584044 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:09.714070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:09.716057 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:09.753024 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1442/2855] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:10.973701 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:10.984137 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee5ea9-c348-483c-97fe-c2e7eb8d4d49 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:10.990984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:10.991784 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:10.992139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:10.992429 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:10.994472 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:11.009593 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:11.011158 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c151aa7-7a2c-4a58-9898-58ca1544ca8a service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:11.024913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1414/2856] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:14.932560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:14.942677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03ea5f-c68b-42d1-80fa-1c5cc646b822 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:14.947059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:14.947721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:14.948011 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:14.948293 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:14.950798 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:14.966928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:14.966928 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:14.978530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1443/2857] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:19.608331 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.608912 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.609021 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.629124 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:19.630444 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:19.637911 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1415/2858] 10.222.0.47 () {58 vars in 967 bytes} [Mon Jul 18 10:53:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:19.650381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:19.662868 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683381b-86a8-4341-8141-1e61d986e545 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:19.667926 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:19.668573 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:19.668894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:19.669530 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:19.677984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:19.679567 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:19.688432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1444/2859] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:53:23.132277 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:23.146042 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7898925-6f3c-4130-8f6d-5c4c983712da None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:23.150555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:23.151458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:23.151907 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:23.152350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] PATH_INFO: `/v3/users/3e52648bc557407db1f2e44727fbde32` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:23.153633 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e52648bc557407db1f2e44727fbde32)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:23.155541 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:23.399139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e52648bc557407db1f2e44727fbde32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:23.418365 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e52648bc557407db1f2e44727fbde32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:23.427475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e52648bc557407db1f2e44727fbde32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:23.563665 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e52648bc557407db1f2e44727fbde32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:23.564565 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e40e3e56-4f7b-4620-89b9-d53dc6d23ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e52648bc557407db1f2e44727fbde32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:23.574010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1416/2860] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:23 2022] DELETE /identity/v3/users/3e52648bc557407db1f2e44727fbde32 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:23.840230 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:23.851754 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bc85c-9971-418f-ad20-14d50ccf2826 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:23.891395 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:23.891688 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:23.891967 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:23.921672 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] MFA Rules not processed for user `cf3fce0b778f46df844a6be985c28b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:23.926734 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:23.932106 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:23.948298 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:23.949815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:23.950118 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:23.950389 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] PATH_INFO: `/v3/projects/a8c3be0d7a0c46d790a0eefd632a6a83` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:23.951626 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c3be0d7a0c46d790a0eefd632a6a83)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:23.953840 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:23.982673 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:24.004949 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1417/2861] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:53:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:24.064787 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c3be0d7a0c46d790a0eefd632a6a83', 'request_id': 'req-01390399-a2a9-49b8-9c84-e03b74023b54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.077992 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c3be0d7a0c46d790a0eefd632a6a83', 'request_id': 'req-01390399-a2a9-49b8-9c84-e03b74023b54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.164517 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.166107 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:24.181361 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02ac5c-264b-4e38-9619-ffc9e02bd65c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.198286 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c3be0d7a0c46d790a0eefd632a6a83', 'request_id': 'req-01390399-a2a9-49b8-9c84-e03b74023b54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.210372 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] The token cache is being invalidate because project a8c3be0d7a0c46d790a0eefd632a6a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:24.211242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.212197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01390399-a2a9-49b8-9c84-e03b74023b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.214308 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1445/2862] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:23 2022] DELETE /identity/v3/projects/a8c3be0d7a0c46d790a0eefd632a6a83 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:24.220568 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8b54f-9e30-443d-aeda-b12f3b9f3b4f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.221692 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8b54f-9e30-443d-aeda-b12f3b9f3b4f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.222942 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8b54f-9e30-443d-aeda-b12f3b9f3b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.257917 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f3d8b54f-9e30-443d-aeda-b12f3b9f3b4f None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:24.259189 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3d8b54f-9e30-443d-aeda-b12f3b9f3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:24.335401 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1446/2863] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:24.342983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.343270 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.343560 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.365893 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:24.367123 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:24.380770 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1447/2864] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:24.381863 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.383737 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.384648 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.384963 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.385239 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] PATH_INFO: `/v3/users/3cd9bc14a25846189a81c3a6845fd551` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.386470 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd9bc14a25846189a81c3a6845fd551)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.388619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.391821 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd86d06-a21f-4fe2-8d03-62a9743dfe6c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.396386 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.397074 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.397364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.397650 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.398385 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:53:24.398696 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.400335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.412498 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1448/2865] 10.222.0.47 () {58 vars in 1152 bytes} [Mon Jul 18 10:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:53:24.416633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.427944 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729536e-a088-46aa-ba00-d175e32d3f8d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.432805 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.433997 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.434274 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.434562 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.435257 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.437242 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.556983 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1449/2866] 10.222.0.47 () {60 vars in 1152 bytes} [Mon Jul 18 10:53:24 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:24.558676 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.586465 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:24.615327 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd9bc14a25846189a81c3a6845fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.629468 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd9bc14a25846189a81c3a6845fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.630292 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08e9f-e3e4-4435-b914-f724f8a81706 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.637603 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.638781 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd9bc14a25846189a81c3a6845fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.638903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.638903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.639104 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.639761 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.641738 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.717896 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd9bc14a25846189a81c3a6845fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.718644 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd9bc14a25846189a81c3a6845fd551'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:24.725556 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1418/2867] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:24 2022] DELETE /identity/v3/users/3cd9bc14a25846189a81c3a6845fd551 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:24.729383 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1450/2868] 10.222.0.47 () {60 vars in 1143 bytes} [Mon Jul 18 10:53:24 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:53:24.731816 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.738031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:24.806677 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7acad-5e3e-4df1-bfa9-98a39c8138a9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.811386 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.812091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.812371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.812647 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.813407 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.815053 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.826458 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1419/2869] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:24.831143 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.844841 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275e8d0-f5e7-435f-a24c-e46413fd5d7b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.849832 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.850659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.851012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.851388 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.852290 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.854031 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:24.866433 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1451/2870] 10.222.0.47 () {58 vars in 1121 bytes} [Mon Jul 18 10:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:53:24.870335 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:24.875569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:24.881017 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589155d-36ff-428f-9250-2e4596ca1912 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:24.885824 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa7797-bec1-4721-8547-cd59358d2090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:24.886741 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa7797-bec1-4721-8547-cd59358d2090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:24.887028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa7797-bec1-4721-8547-cd59358d2090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:24.887306 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa7797-bec1-4721-8547-cd59358d2090 admin admin] PATH_INFO: `/v3/projects/7e4d5cc5945b4690bbe6ffc5fd23841d/users/4a94d89d0285425c80552cfef8f2a7c0/roles/19944ab23542462988b68290167ac622` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:24.903539 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa7797-bec1-4721-8547-cd59358d2090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4d5cc5945b4690bbe6ffc5fd23841d, user_id=4a94d89d0285425c80552cfef8f2a7c0, role_id=19944ab23542462988b68290167ac622)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:24.906362 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa7797-bec1-4721-8547-cd59358d2090 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:25.045255 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:25.045681 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1420/2871] 10.222.0.47 () {60 vars in 1482 bytes} [Mon Jul 18 10:53:24 2022] PUT /identity/v3/projects/7e4d5cc5945b4690bbe6ffc5fd23841d/users/4a94d89d0285425c80552cfef8f2a7c0/roles/19944ab23542462988b68290167ac622 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:25.050553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:25.056303 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:25.056601 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:25.056937 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:25.065151 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cc7cc-64fe-4008-9581-625751155353 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:25.086334 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] MFA Rules not processed for user `4a94d89d0285425c80552cfef8f2a7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:25.087619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:25.130477 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:25.131356 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:25.131652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:25.131937 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] PATH_INFO: `/v3/projects/6d89e2b7a84b4fb6ba7060088b60f6ae` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:25.133193 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d89e2b7a84b4fb6ba7060088b60f6ae)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:25.135116 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:25.157139 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1421/2872] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:53:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:53:25.168162 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:25.169545 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:25.225216 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58aae1-fb4e-4e96-b067-4a4a69f18498 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:25.228945 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:25.229652 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:25.229995 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:25.230271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:25.241006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:25.241006 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:25.250410 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1422/2873] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:53:25.284432 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d89e2b7a84b4fb6ba7060088b60f6ae', 'request_id': 'req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.291773 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d89e2b7a84b4fb6ba7060088b60f6ae', 'request_id': 'req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.354066 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d89e2b7a84b4fb6ba7060088b60f6ae', 'request_id': 'req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.370766 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] The token cache is being invalidate because project 6d89e2b7a84b4fb6ba7060088b60f6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:25.371400 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.372208 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.374904 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1452/2874] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:25 2022] DELETE /identity/v3/projects/6d89e2b7a84b4fb6ba7060088b60f6ae => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:53:25.378668 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:25.393486 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a875e-5946-4b60-bef8-6f0acdf219a3 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:25.455164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:25.456045 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:25.456333 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:25.456614 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] PATH_INFO: `/v3/users/abbd7c912b5848feb83c7335852006de` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:25.457857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbd7c912b5848feb83c7335852006de)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:25.459811 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:25.472301 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:25.738653 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbd7c912b5848feb83c7335852006de'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.750984 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbd7c912b5848feb83c7335852006de'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.758078 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbd7c912b5848feb83c7335852006de'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.860220 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbd7c912b5848feb83c7335852006de'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.860878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbd7c912b5848feb83c7335852006de'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:25.867609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1423/2875] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:25 2022] DELETE /identity/v3/users/abbd7c912b5848feb83c7335852006de => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:26.147572 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:26.149331 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:26.168881 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999f3f4-7b6b-4fd2-bfc5-67c4cf784f9c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:26.217697 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:26.244990 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:26.245894 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:26.246181 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:26.246457 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] PATH_INFO: `/v3/projects/a0f8905c13a84c4ba3271019fdf189d2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:26.247665 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f8905c13a84c4ba3271019fdf189d2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:26.249613 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:26.438381 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f8905c13a84c4ba3271019fdf189d2', 'request_id': 'req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.445314 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f8905c13a84c4ba3271019fdf189d2', 'request_id': 'req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.546263 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f8905c13a84c4ba3271019fdf189d2', 'request_id': 'req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.557070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] The token cache is being invalidate because project a0f8905c13a84c4ba3271019fdf189d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:26.557444 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.558322 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:26.559866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1453/2876] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:26 2022] DELETE /identity/v3/projects/a0f8905c13a84c4ba3271019fdf189d2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:35.883173 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:36.218645 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:36.221296 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:36.296250 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbecb4-a11f-4acf-ad96-e6e9e5b6d8fa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:36.346263 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:36.346829 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:36.347119 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:36.347409 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:36.349232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:36.359387 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:36.361182 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39199323-3b07-4787-8a63-3457775373e2 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:36.370205 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1424/2877] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:36.586850 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:37.230306 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:37.232070 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:37.311721 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e861cd-aad9-4d7e-b3a5-19f2f2124cd5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:37.316659 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:37.317760 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:37.318046 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:37.318331 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] PATH_INFO: `/v3/users/4a94d89d0285425c80552cfef8f2a7c0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:37.319543 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a94d89d0285425c80552cfef8f2a7c0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:37.321507 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:37.544624 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a94d89d0285425c80552cfef8f2a7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.551652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a94d89d0285425c80552cfef8f2a7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.654486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a94d89d0285425c80552cfef8f2a7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.655144 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a94d89d0285425c80552cfef8f2a7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.655771 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a94d89d0285425c80552cfef8f2a7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:37.676152 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1454/2878] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:37 2022] DELETE /identity/v3/users/4a94d89d0285425c80552cfef8f2a7c0 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:37.680721 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:37.681197 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:37.681318 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:37.699211 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] MFA Rules not processed for user `02055864266e49b39ebadd610e2d403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:37.701184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:37.768437 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1425/2879] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:37.780115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:37.781833 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:37.866165 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c52df-7cc0-44d5-b8d6-49fafd6c8907 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:37.875376 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:37.876078 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:37.876370 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:37.877080 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:37.887794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:37.889319 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:37.908884 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1455/2880] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:38.562134 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:38.563805 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:38.575857 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528b7b-66ab-4aa7-a4ad-b0ea5b3dd532 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:38.580960 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:38.582732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:38.582732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:38.582732 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] PATH_INFO: `/v3/projects/7e4d5cc5945b4690bbe6ffc5fd23841d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:38.583934 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4d5cc5945b4690bbe6ffc5fd23841d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:38.585976 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:38.724619 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4d5cc5945b4690bbe6ffc5fd23841d', 'request_id': 'req-3f0446cc-db21-4823-9943-efc81dd0896b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.730043 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:38.843278 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4d5cc5945b4690bbe6ffc5fd23841d', 'request_id': 'req-3f0446cc-db21-4823-9943-efc81dd0896b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.849371 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4d5cc5945b4690bbe6ffc5fd23841d', 'request_id': 'req-3f0446cc-db21-4823-9943-efc81dd0896b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.861273 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] The token cache is being invalidate because project 7e4d5cc5945b4690bbe6ffc5fd23841d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:38.861569 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.862251 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f0446cc-db21-4823-9943-efc81dd0896b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:38.863807 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1426/2881] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:38 2022] DELETE /identity/v3/projects/7e4d5cc5945b4690bbe6ffc5fd23841d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:43.277657 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:43.279994 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:43.309815 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:43.359456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abbce9-85f3-4a58-b875-5300a482b1e3 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:43.428131 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:43.429085 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:43.429476 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:43.429902 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:43.432182 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:43.448518 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:43.450250 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:43.463327 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1456/2882] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:44.489210 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:44.489676 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:44.489919 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:44.538252 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] MFA Rules not processed for user `8721bfc5b53a421d99d1e75f84e639b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:53:44.539506 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:44.608527 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1427/2883] 10.222.0.47 () {58 vars in 967 bytes} [Mon Jul 18 10:53:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:53:44.621271 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:44.622631 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:44.745652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec70c1b-255f-4fbf-9fdb-b701c9ebde98 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:44.751576 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385713e7-b247-4631-bc39-105836671900 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5554e4999dc04da682c7f1cbd5f75abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:44.752752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385713e7-b247-4631-bc39-105836671900 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:44.752752 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385713e7-b247-4631-bc39-105836671900 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:44.753091 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385713e7-b247-4631-bc39-105836671900 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:44.761646 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385713e7-b247-4631-bc39-105836671900 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:44.763232 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385713e7-b247-4631-bc39-105836671900 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:44.771569 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1457/2884] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Jul 18 10:53:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:53:49.710761 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:49.714455 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.792547 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31556-88c8-4f4d-9db1-2bd72db7cb00 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:49.797837 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:49.799392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:49.799392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:49.799392 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:49.800962 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:49.813027 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:49.814631 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:49.824555 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1428/2885] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:53:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:53:53.454577 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385713e7-b247-4631-bc39-105836671900 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:53.456819 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-385713e7-b247-4631-bc39-105836671900 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:53.475364 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:53.521874 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385713e7-b247-4631-bc39-105836671900 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:53.526517 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:53.527424 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:53.527720 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:53.528002 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] PATH_INFO: `/v3/users/4ad0be39d4374c6abb43421da6d2c722` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:53.529903 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad0be39d4374c6abb43421da6d2c722)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:53.531859 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:53.688456 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad0be39d4374c6abb43421da6d2c722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:53.695178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad0be39d4374c6abb43421da6d2c722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:53.766677 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad0be39d4374c6abb43421da6d2c722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:53.767316 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad0be39d4374c6abb43421da6d2c722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:53.767943 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad0be39d4374c6abb43421da6d2c722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:53.785793 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1458/2886] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:53 2022] DELETE /identity/v3/users/4ad0be39d4374c6abb43421da6d2c722 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:54.055151 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:54.067071 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02ee56-4521-40ea-ae76-cfe2e9ad3488 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:54.114120 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:54.128950 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:54.129814 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:54.130100 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:54.130381 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] PATH_INFO: `/v3/projects/6754afdf4fcc46fbb4014f38ae3d9b73` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:54.131618 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6754afdf4fcc46fbb4014f38ae3d9b73)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:54.133656 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:54.272184 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6754afdf4fcc46fbb4014f38ae3d9b73', 'request_id': 'req-3f61ce17-f5fc-4aad-929a-607e9b89eba0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.367148 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6754afdf4fcc46fbb4014f38ae3d9b73', 'request_id': 'req-3f61ce17-f5fc-4aad-929a-607e9b89eba0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.373983 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6754afdf4fcc46fbb4014f38ae3d9b73', 'request_id': 'req-3f61ce17-f5fc-4aad-929a-607e9b89eba0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.385060 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] The token cache is being invalidate because project 6754afdf4fcc46fbb4014f38ae3d9b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:54.385346 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.385968 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.387475 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1429/2887] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:54 2022] DELETE /identity/v3/projects/6754afdf4fcc46fbb4014f38ae3d9b73 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:54.391652 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:54.393122 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:54.406100 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca929-3a09-4398-b9f4-0fa188543264 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:54.461866 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:54.462790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:54.463083 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:54.463366 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] PATH_INFO: `/v3/users/46e22ee29b654193853528f5d323e759` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:54.464701 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e22ee29b654193853528f5d323e759)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:54.466789 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:54.631102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:53:54.639283 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e22ee29b654193853528f5d323e759'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.646030 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e22ee29b654193853528f5d323e759'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.699380 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e22ee29b654193853528f5d323e759'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.700006 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e22ee29b654193853528f5d323e759'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.700625 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e22ee29b654193853528f5d323e759'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:54.720531 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1459/2888] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:53:54 2022] DELETE /identity/v3/users/46e22ee29b654193853528f5d323e759 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:53:54.979344 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:53:54.980878 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:53:54.994198 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ce17-f5fc-4aad-929a-607e9b89eba0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:53:55.052144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:53:55.053034 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:53:55.053319 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:53:55.053595 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] PATH_INFO: `/v3/projects/9a85faa949064966922e9e4ca9d341f1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:53:55.054769 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a85faa949064966922e9e4ca9d341f1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:53:55.056660 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:53:55.225978 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a85faa949064966922e9e4ca9d341f1', 'request_id': 'req-668cd162-463d-4faa-9ef8-3b8c238add93'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:55.317115 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a85faa949064966922e9e4ca9d341f1', 'request_id': 'req-668cd162-463d-4faa-9ef8-3b8c238add93'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:55.323241 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a85faa949064966922e9e4ca9d341f1', 'request_id': 'req-668cd162-463d-4faa-9ef8-3b8c238add93'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:55.335450 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] The token cache is being invalidate because project 9a85faa949064966922e9e4ca9d341f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:53:55.335743 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:55.336380 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-668cd162-463d-4faa-9ef8-3b8c238add93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:53:55.337940 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1430/2889] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:53:54 2022] DELETE /identity/v3/projects/9a85faa949064966922e9e4ca9d341f1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:54:04.734278 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:54:05.332427 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:27.925551 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:27.926973 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:27.985261 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2f3b8-4029-4499-b523-8ed29e1f65a9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:28.042310 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8f56a4b2d94ec1aec6f0d044cbc1bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:28.042889 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:28.043178 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:28.043460 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:28.045666 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:28.105053 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:28.106486 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:28.115795 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1460/2890] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:33.067871 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:33.068414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:33.068414 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:33.088154 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] MFA Rules not processed for user `cf3fce0b778f46df844a6be985c28b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:55:33.095913 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:33.096269 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1431/2891] 10.222.0.47 () {58 vars in 957 bytes} [Mon Jul 18 10:55:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:55:33.410218 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:33.411739 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:33.467640 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8dcf-d63b-4657-bf62-fb2bf482b43c service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:33.471197 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:33.471836 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:33.472120 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:33.472397 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:33.474168 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:33.484196 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:33.485633 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:33.493699 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1461/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:35.202018 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:35.203543 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:35.276326 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad424b0d-c87c-4b67-8f91-13616d0d65c9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:35.281350 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:35.282255 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:35.282434 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:35.282710 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:35.284554 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:35.296044 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:35.297643 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:35.307918 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1432/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:43.490034 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:45.303825 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=35 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:55:48.761790 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:55:48.763258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:48.839180 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8679f7-7032-4e06-9d9b-616b98cbf2d1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:55:48.844782 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1f5d0bba8a4178977c19b7503ef6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:55:48.845452 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:55:48.845724 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:55:48.846020 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:55:48.847794 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:55:48.861115 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:55:48.865861 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:55:48.874514 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1462/2894] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:55:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:55:58.869553 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=34 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:11.246088 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:11.247509 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:11.261209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad385324-d58f-4abc-adfb-42485c390392 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:11.266501 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:11.267187 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:11.267485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:11.267758 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:11.270504 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:11.285801 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:11.287398 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:11.297396 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1433/2895] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:21.293083 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:47.337656 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:47.347984 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c29c01-32f5-4617-ac6b-073e8fd0c938 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:47.351616 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0633dadc8b43c5a13f52cace5bb85d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:47.352259 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:47.352548 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:47.352867 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:47.354804 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:47.422200 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:47.424549 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:47.434258 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1463/2896] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:51.166587 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:51.178600 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0460ebd-45a0-40d6-89ef-88d8ed233e52 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:51.183762 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:51.184560 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:51.184998 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:51.185299 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:51.187166 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:51.199125 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:51.201096 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:51.211144 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1434/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 10:57:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:57:55.329102 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:55.330901 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:55.345464 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e06df-ad3a-4cc2-990a-1b6846014a42 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:55.350176 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:55.351048 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:55.351332 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:55.351610 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] PATH_INFO: `/v3/users/cf3fce0b778f46df844a6be985c28b09` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:55.352857 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3fce0b778f46df844a6be985c28b09)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:55.354779 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:55.614751 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3fce0b778f46df844a6be985c28b09'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:55.621732 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3fce0b778f46df844a6be985c28b09'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:55.709664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3fce0b778f46df844a6be985c28b09'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:55.710304 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3fce0b778f46df844a6be985c28b09'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:55.710923 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3fce0b778f46df844a6be985c28b09'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:55.729012 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1464/2898] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:57:55 2022] DELETE /identity/v3/users/cf3fce0b778f46df844a6be985c28b09 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:57:55.985698 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:55.999485 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d477d9-0037-4665-881c-0d4c1128fd22 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:56.058675 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:56.059532 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:56.059812 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:56.060086 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] PATH_INFO: `/v3/projects/5d7ee0634e694265868916311d7fb4f1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:56.061271 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7ee0634e694265868916311d7fb4f1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:56.063232 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:56.310908 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7ee0634e694265868916311d7fb4f1', 'request_id': 'req-6c773a77-6cf3-4756-8311-10a183dd3646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.464590 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7ee0634e694265868916311d7fb4f1', 'request_id': 'req-6c773a77-6cf3-4756-8311-10a183dd3646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.471010 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7ee0634e694265868916311d7fb4f1', 'request_id': 'req-6c773a77-6cf3-4756-8311-10a183dd3646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.482390 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] The token cache is being invalidate because project 5d7ee0634e694265868916311d7fb4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:57:56.482772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.483438 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.485863 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1435/2899] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:57:55 2022] DELETE /identity/v3/projects/5d7ee0634e694265868916311d7fb4f1 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:57:56.489964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:56.491405 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:56.505300 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfb390-e7ef-472e-8620-cd7d73b9ad9c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:56.560494 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:56.561399 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:56.561684 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:56.561964 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] PATH_INFO: `/v3/users/f175c299dfff4c8882c5c0c2657eb528` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:56.563735 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f175c299dfff4c8882c5c0c2657eb528)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:56.565737 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:56.571059 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:56.769949 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f175c299dfff4c8882c5c0c2657eb528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.776664 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f175c299dfff4c8882c5c0c2657eb528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.876134 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f175c299dfff4c8882c5c0c2657eb528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.876838 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f175c299dfff4c8882c5c0c2657eb528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.877463 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f175c299dfff4c8882c5c0c2657eb528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:56.896335 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1465/2900] 10.222.0.47 () {58 vars in 1223 bytes} [Mon Jul 18 10:57:56 2022] DELETE /identity/v3/users/f175c299dfff4c8882c5c0c2657eb528 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:57:57.267609 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:57:57.269493 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:57:57.283164 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773a77-6cf3-4756-8311-10a183dd3646 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:57:57.340558 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02055864266e49b39ebadd610e2d403c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a98445bbc6b4a039db83aaf6d202161', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:57:57.341454 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:57:57.341740 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:57:57.342028 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] PATH_INFO: `/v3/projects/d3126c02b94741949dc193f9d0508415` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:57:57.343209 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3126c02b94741949dc193f9d0508415)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:57:57.345778 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:57:57.366091 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:57:57.532783 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3126c02b94741949dc193f9d0508415', 'request_id': 'req-0199be39-61a1-47e1-8d92-3281a8fb0fbd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:57.612635 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3126c02b94741949dc193f9d0508415', 'request_id': 'req-0199be39-61a1-47e1-8d92-3281a8fb0fbd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:57.618724 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3126c02b94741949dc193f9d0508415', 'request_id': 'req-0199be39-61a1-47e1-8d92-3281a8fb0fbd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:57.630482 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] The token cache is being invalidate because project d3126c02b94741949dc193f9d0508415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:57:57.630772 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:57.631423 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0199be39-61a1-47e1-8d92-3281a8fb0fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:57:57.633031 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1436/2901] 10.222.0.47 () {58 vars in 1232 bytes} [Mon Jul 18 10:57:57 2022] DELETE /identity/v3/projects/d3126c02b94741949dc193f9d0508415 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:58:06.892812 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:58:07.647510 dv-cinder-846171-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 11:02:18.248802 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 11:02:18.250231 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 11:02:18.391615 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785bda0-541d-4410-8ec9-e9c74aa33285 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 11:02:18.449141 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7b860ca751f4eb7bcc72617e05abd30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12759b13d97d46d692472e8cd8550ab2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 11:02:18.450056 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 11:02:18.450342 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 11:02:18.450623 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 11:02:18.452420 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 11:02:18.464499 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 11:02:18.466107 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9eb5f7-4e65-4d97-aa33-462700518084 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 11:02:18.476412 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1466/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Jul 18 11:02:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 11:02:28.471728 dv-cinder-846171-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}